Array Cameras Incorporating Optics with Modulation Transfer Functions Greater than Sensor Nyquist Frequency for Capture of Images used in Super-Resolution Processing

ABSTRACT

A variety of optical arrangements and methods of modifying or enhancing the optical characteristics and functionality of these optical arrangements are provided. The optical arrangements being specifically designed to operate with camera arrays that incorporate an imaging device that is formed of a plurality of imagers that each include a plurality of pixels. The plurality of imagers include a first imager having a first imaging characteristics and a second imager having a second imaging characteristics. The images generated by the plurality of imagers are processed to obtain an enhanced image compared to images captured by the imagers. In many optical arrangements the MTF characteristics of the optics allow for contrast at spatial frequencies that are at least as great as the desired resolution of the high resolution images synthesized by the array camera, and significantly greater than the Nyquist frequency of the pixel pitch of the pixels on the focal plane, which in some cases may be 1.5, 2 or 3 times the Nyquist frequency.

RELATED APPLICATION

This application is a continuation of U.S. Non-Provisional patentapplication Ser. No. 13/832,120, entitled “Optical Arrangements for Usewith an Array Camera”, filed Mar. 15, 2013, which application is aContinuation in part of U.S. Non-Provisional patent application Ser. No.13/536,520, entitled “Optical Arrangements for Use with an ArrayCamera”, which issued Jan. 3, 2013 as U.S. Pat. No. 8,804,255, whichclaims priority to U.S. Provisional Application No. 61/502,158 filedJun. 28, 2011. The disclosures of these applications are incorporatedherein by reference.

FIELD OF THE INVENTION

The present invention is related to novel optical arrangements, designsand elements for use in an array camera, and more specifically tooptical arrangements of varying configurations having modulationtransfer function (MTF) characteristics capable of implementing superresolution for use with arrays of image sensors.

BACKGROUND OF THE INVENTION

Image sensors are used in cameras and other imaging devices to captureimages. In a typical imaging device, light enters at one end of theimaging device and is directed to an image sensor by an optical elementsuch as a lens. In most imaging devices, one or more layers of opticalelements are placed before and after the aperture stop to focus lightonto the image sensor. Recently array cameras having many imagers andlenses have been developed. In most cases, multiple copies of theoptical elements must be formed laterally for use in array cameras.

Conventionally, optical arrays can be formed by molding or embossingfrom a master lens array, or fabricated by standard lithographic orother means. However, the standard polymer-on-glass WLO and monolithiclens WLO manufacturing techniques have so far not been adapted for thespecific high performance requirements of array cameras. In particular,some technical limitations of conventional WLO-processes need to bereduced, such as, for example, minimum substrate thickness requirements,inflexibility of where to place the aperture stop, accuracy, etc. Theflexibility of such choices or processes needs to be increased in orderto meet the high demands by array cameras otherwise such WLO techniquescannot be used to manufacture array cameras. Accordingly, a need existsfor fabrication processes capable of accurately forming these arrays andfor optical arrangements that give an increased flexibility inmanufacturing so that the image processing software of these new typesof array-type cameras can take advantage to deliver superior imagequality at the system level.

The optical transfer function (OTF) of an imaging system (camera, videosystem, microscope etc.) is considered the true measure of an imagingsystem's performance, i.e., the resolution (minimum feature size ormaximum spatial frequency that can be imaged with sufficient contrast)or image sharpness (the contrast at a given spatial frequency)obtainable by an imaging system. While optical resolution, as commonlyused with reference to camera systems, describes only the number ofpixels in an image, and hence the potential to show fine detail, thetransfer function describes the ability of adjacent pixels to changefrom black to white in response to patterns of varying spatialfrequency, and hence the actual capability to show fine detail, whetherwith full or reduced contrast. The optical transfer Function (OTF)consists of two components: the modular transfer function (MTF), whichis the magnitude of the OTF, and the phase transfer function (PTF),which is the phase component.

In cameras, the MTF is the most relevant measurement of performance, andis generally taken as an objective measurement of the ability of anoptical system to transfer various levels of detail from an object to animage. The MTF is measured in terms of contrast (degrees of gray), or ofmodulation, produced from a perfect source of that detail level (thus itis the ratio of contrast between the object and the image). The amountof detail in an image is given by the resolution of the optical system,and is customarily specified in line pairs per millimeter (Ip/mm). Aline pair is one cycle of a light bar and dark bar of equal width andhas a contrast of unity. Contrast is defined as the ratio of thedifference in maximum intensity (I_(max)) and minimum intensity(I_(min)) over the sum of I_(max) and I_(min), where I_(max) is themaximum intensity produced by an image (white) and I_(min) is theminimum intensity (black). The MTF then is the plot of contrast,measured in percent, against spatial frequency measured in Ip/rm. Thisgraph is customarily normalized to a value of 1 at zero spatialfrequency (all white or black).

SUMMARY

The current invention is directed to optical arrangements for use withan array of cameras where the MTF of each of the optical arrangements orstacks for each camera of the array of cameras is at least as high asthe desired MTF of the super resolution image synthesized from thecombined images of the of cameras of the camera array.

In many embodiments, the camera array includes a plurality of cameras,where each camera includes a separate optics, and a plurality of lightsensing elements, and each camera is configured to independently capturean image of a scene;

wherein the optics of each camera are configured so that each camera hasa field of view that is shifted with respect to the field-of-views ofthe other cameras so that each shift includes a sub-pixel shifted viewof the scene;

wherein the light sensing elements have a pixel pitch defining a Nyquistfrequency, and where the optics of each camera have a modular transferfunction (MTF) such that the optics optically resolve, with sufficientcontrast, spatial frequencies larger than the Nyquist frequency (Ny);

wherein the camera array is a monolithic integrated module comprising asingle semiconductor substrate on which all of the sensor elements areformed, and optics including a plurality of lens elements, where eachlens element forms part of the separate optics for one of the cameras;

wherein each of the cameras includes one of a plurality of differenttypes of filer; and

wherein cameras having the same type of filter are uniformly distributedabout the geometric center of the camera array.

In other embodiments the camera array, includes:

a plurality of cameras, where each camera includes a separate optics,and a plurality of light sensing elements each having a pixel pitchdefining a Nyquist frequency (Ny), and each camera is configured toindependently capture a low resolution image of a scene;

a processor configured to synthesize a higher resolution image from theplurality of lower resolution images, the high resolution image has acharacteristic MTF;

wherein the optics of each camera are configured so that each camera hasa field of view that is shifted with respect to the field-of-views ofthe other cameras so that each shift includes a sub-pixel shifted viewof the scene;

wherein the optics of each camera have a modular transfer function (MTF)at least as large of the desired MTF of the high resolution image;

wherein the camera array is a monolithic integrated module comprising asingle semiconductor substrate on which all of the sensor elements areformed, and optics including a plurality of lens elements, where eachlens element forms part of the separate optics for one of the cameras;

wherein each of the cameras includes one of a plurality of differenttypes of filer; and wherein cameras having the same type of filter areuniformly distributed about the geometric center of the camera array.

In still other embodiments, the cut-off MTF of the optics is at least1.5 times the Ny, at least 2 times the Ny, or at least 3 times the Ny.

In yet other embodiments the optics of each camera include athree-surface optical arrangement includes:

-   -   a first lens element having a first convex proximal surface and        a first concave distal surface, wherein the diameter of the        first convex surface is larger than the diameter of the first        concave surface;    -   a second lens element having a substantially flat second        proximal surface and a second convex distal surface, wherein the        diameter of the flat second proximal surface is smaller than the        diameter of the second convex surface, and wherein the diameter        of the second convex surface is intermediate between the        diameters of the first convex surface and the first concave        surface; and    -   wherein the first and second lens elements are arranged        sequentially in optical alignment with an imager positioned at        the distal end thereof.

In still yet other embodiments the optics of each camera include afive-surface optical arrangement including:

-   -   a first lens element having a first convex proximal surface and        a first concave distal surface, wherein the diameter of the        first convex surface is larger than the diameter of the first        concave surface;    -   a second lens element having a second concave proximal surface        and a second convex distal surface, wherein the diameter of the        second concave proximal surface is smaller than the diameter of        the second convex surface;    -   a third lens element having a third concave proximal surface and        a third planar distal surface, wherein the diameter of the third        concave proximal surface is larger than the diameters of any of        the surfaces of the first and second lens elements; and    -   wherein the first, second and thirds lens elements are arranged        sequentially in optical alignment with an imager positioned at        the distal end thereof.

In still yet other embodiments the optics of each camera include asubstrate embedded hybrid lens optical arrangement including:

-   -   a substrate having proximal and distal sides;    -   a first monolithic lens element having first proximal and distal        surfaces disposed on the proximal side of said substrate;    -   a second monolithic lens element having second proximal and        distal surfaces disposed on the distal side of said substrate;    -   at least one aperture disposed on said substrate in optical        alignment with said first and second lens elements; and    -   wherein the first and second lens elements are arranged        sequentially in optical alignment with an imager positioned at        the distal end thereof.

In still yet other embodiments the optics of each camera include amonolithic lens optical arrangement including:

-   -   at least one lens element comprising:    -   a first monolithic lens having first proximal and distal        surfaces, wherein the first proximal surface of the first        monolithic lens has one of either a concave or convex profile,        and wherein the first distal surface of the first monolithic        lens has a plano profile;    -   at least one aperture disposed on the first distal surface of        the first monolithic lens and in optical alignment therewith;    -   a second monolithic lens having second proximal and distal        surfaces, wherein the second proximal surface of the second        monolithic lens has a plano profile, and wherein the second        distal surface of the second monolithic lens has one of either a        concave or convex profile, and wherein the second monolithic        lens is arranged in optical alignment with said aperture; and    -   wherein the first monolithic lens element is in direct contact        with the aperture and the second monolithic lens.

In still yet other embodiments, the optics of each camera include athree-element monolithic lens optical arrangement including:

-   -   a first lens element having a first convex proximal surface and        a first plano distal surface;    -   a second lens element having a second concave proximal surface        and a second convex distal surface;    -   a third menisci lens element having a third concave proximal        surface and a third convex distal surface;    -   at least one aperture disposed on the first plano distal        surface; and    -   wherein the first, second and third lens elements are arranged        sequentially in optical alignment with the aperture stop and an        imager.

In one embodiment, the invention is directed to a three-surface opticalarrangement for an array camera. In such an embodiment, the opticalarrangement includes:

-   -   a first lens element having a first convex proximal surface and        a first concave distal surface, where the diameter of the first        convex surface is larger than the diameter of the first concave        surface,    -   a second lens element having a substantially flat second        proximal surface and a second convex distal surface, where the        diameter of the flat second proximal surface is smaller than the        diameter of the second convex surface, and where the diameter of        the second convex surface is intermediate between the diameters        of the first convex surface and the first concave surface; and    -   wherein the first and second lens elements are arranged        sequentially in optical alignment with an imager positioned at        the distal end thereof.

In one embodiment of the three-surface optical arrangement, the surfacesof the first element are separated by a first substrate, and thesurfaces of the second element are separated by a second substrate. Inanother such embodiment, the flat second proximal surface is formed bythe second substrate. In still another such embodiment, an aperture stopis disposed on the flat second proximal surface. In yet another suchembodiment, at least one aperture is disposed on at least one of thefirst or second substrates. In still yet another such embodiment, anaperture structure is disposed between said first and second lenselements, comprising at least one aperture substrate having at least oneaperture disposed thereon. In still yet another such embodiment, thefirst and second lens elements and the second lens element and theimager are separated by spacers. In still yet another such embodiment, afilter is disposed on at least one of the first or second substrates. Instill yet another such embodiment, at least two of the surfaces of thelens elements are formed from materials having different Abbe-numbers.In still yet another such embodiment, the convex surfaces are formedfrom crown-like materials, and the concave surfaces are formed fromflint-like materials.

In another embodiment of the three-surface optical arrangement, an arrayof such arrangements are described, where the array is designed to imagea selected wavelength band, and where the profile of at least one of thelens surfaces within each optical arrangement is adapted to optimallyimage only a narrow-band portion of the selected wavelength band suchthat in combination the plurality of arrangements within the array imagethe entirety of the selected wavelength band.

In another embodiment, the invention is directed to a five-surfaceoptical arrangement for an array camera. In such an embodiment, theoptical arrangement includes:

-   -   a first lens element having a first convex proximal surface and        a first concave distal surface, where the diameter of the first        convex surface is larger than the diameter of the first concave        surface;    -   a second lens element having a second concave proximal surface        and a second convex distal surface, where the diameter of the        second concave proximal surface is smaller than the diameter of        the second convex surface;    -   a third lens element having a third concave proximal surface and        a third planar distal surface, where the diameter of the third        concave proximal surface is larger than the diameters of any of        the surfaces of the first and second lens elements; and    -   where the first, second and thirds lens elements are arranged        sequentially in optical alignment with an imager positioned at        the distal end thereof.

In one such embodiment of the five-surface optical arrangement thesurfaces of the first element are separated by a first substrate, andthe surfaces of the second element are separated by a second substrate.In another such embodiment, the third planar distal surface is incontact with one of either the image sensor or a cover glass disposedover the image sensor. In still another such embodiment, an aperturestop is disposed on the first concave distal surface. In yet anothersuch embodiment, an aperture stop is disposed on the first substrateadjacent to the first concave distal surface. In yet another suchembodiment, at least one aperture is disposed within the first lenselement. In still yet another such embodiment, an aperture structure isdisposed between at least two of said lens elements, the aperturestructure comprising at least one aperture substrate having at least oneaperture disposed thereon. In still yet another such embodiment, thefirst and second lens elements, and the second and thirds lens elementsare separated by spacers. In still yet another such embodiment, a filteris disposed within at least one of the first and second lens elements.In still yet another such embodiment, at least two of the surfaces ofthe lens elements are formed from materials having differentAbbe-numbers. In still yet another such embodiment, the convex surfacesare formed from crown-like materials, and the concave surfaces areformed from flint-like materials. In still yet another such embodiment,an air-gap is positioned between the third lens element and the imagesensor. In still yet another such embodiment, at least one substrate isdisposed between the surfaces of at least one of the lens elements. Instill yet another such embodiment, a substrate is disposed between thethird lens element and the imager. In still yet another such embodiment,at least one aperture is disposed on at least one substrate within thelens elements. In still yet another such embodiment, at least oneaperture is embedded within the first lens element.

In another embodiment of the five-surface optical arrangement, aplurality of the five-surface optical arrangements is provided in anarray. In such embodiment, the array is designed to image a selectedwavelength band, and wherein the profile of at least one of the lenssurfaces within each optical arrangement is adapted to optimally imageonly a narrow-band portion of the selected wavelength such that incombination the plurality of arrangements within the array image theentirety of the selected wavelength band.

In another embodiment, the invention is directed to a substrate embeddedhybrid lens optical arrangement for an array camera. In such anembodiment, the optical arrangement includes:

-   -   a substrate having proximal and distal sides;    -   a first monolithic lens element having first proximal and distal        surfaces disposed on the proximal side of the substrate;    -   a second monolithic lens element having second proximal and        distal surfaces disposed on the distal side of the substrate;    -   at least one aperture disposed on said substrate in optical        alignment with the first and second lens elements; and    -   wherein the first and second lens elements are arranged        sequentially in optical alignment with an imager positioned at        the distal end thereof.

In another embodiment of the substrate embedded hybrid lens opticalarrangement at least two axially aligned apertures are disposed on saidsubstrate. In another such embodiment, the at least two axially alignedapertures are one of either the same or different sizes. In stillanother such embodiment, at least one coating is disposed on saidsubstrate in optical alignment with said at least one aperture. In yetanother such embodiment, the at least one coating is selected from thegroup consisting of a polarization filter, a color filter, an IRCFfilter, and a NIR-pass filter. In still yet another such embodiment, thesubstrate is formed from a material that acts as a filter selected fromthe group consisting of a polarization filter, a color filter, an IRCFfilter, and a NIR-pass filter. In still yet another such embodiment, thesubstrate further comprises an adaptive optical element. In still yetanother such embodiment, at least two of the lens elements are formedfrom materials having different Abbe-numbers.

In still another embodiment of the substrate embedded hybrid lensoptical arrangement, such an arrangement is part of a wafer stackcomprising a plurality of the substrate embedded hybrid lens opticalarrangements including:

-   -   a plurality of wafer surfaces formed from the elements of the        arrangements; and    -   at least two alignment marks formed in relation to each wafer        surface, each of said alignment marks being cooperative with an        alignment mark on an adjacent wafer surface such that said        alignment marks when cooperatively aligned aide in the lateral        and rotational alignment of the lens surfaces with the        corresponding apertures.

In yet another embodiment of the substrate embedded hybrid lens opticalarrangement, the arrangement is part of an array comprising a pluralityof the substrate embedded hybrid lens optical arrangements, where thearray is designed to image a selected wavelength band, and wherein theprofile of at least one of the lens surfaces within each opticalarrangement is adapted to optimally image only a narrow-band portion ofthe selected wavelength such that in combination the plurality ofarrangements within the array image the entirety of the selectedwavelength band.

In still another embodiment, the invention is directed to a monolithiclens optical arrangement for an array camera. In such an embodiment, theoptical arrangement includes:

-   -   at least one lens element itself comprising:    -   a first monolithic lens having first proximal and distal        surfaces, where the first proximal surface of the first        monolithic lens has one of either a concave or convex profile,        and where the first distal surface of the first monolithic lens        has a plano profile;    -   at least one aperture disposed on the first distal surface of        the first monolithic lens and in optical alignment therewith;    -   a second monolithic lens having second proximal and distal        surfaces, where the second proximal surface of the second        monolithic lens has a plano profile, and where the second distal        surface of the second monolithic lens has one of either a        concave or convex profile, and where the second monolithic lens        is arranged in optical alignment with said aperture; and    -   where the first monolithic lens element is in direct contact        with the aperture and the second monolithic lens.

In another embodiment the monolithic optical arrangement includes atleast one filter disposed on said plano surface in optical alignmentwith said at least one aperture. In still another such embodiment, themonolithic lenses are formed from materials having differentAbbe-numbers. In yet another such embodiment, at least two lens elementsare formed.

In still another embodiment the monolithic lens optical arrangement ispart of an array comprising a plurality of the monolithic opticalarrangements, where the array is designed to image a selected wavelengthband, and wherein the profile of at least one of the lens surfaceswithin each optical arrangement is adapted to optimally image only anarrow-band portion of the selected wavelength such that in combinationthe plurality of arrangements within the array image the entirety of theselected wavelength band.

In yet another embodiment of the monolithic lens optical arrangement,such an arrangement is part of a wafer stack comprising:

-   -   a plurality of wafer surfaces formed from the elements of the        arrangements; and    -   at least two alignment marks formed in relation to each wafer        surface, each of the alignment marks being cooperative with an        alignment mark on an adjacent wafer surface such that the        alignment marks when cooperatively aligned aide in the lateral        and rotational alignment of the lens surfaces with the        corresponding apertures.

In yet another embodiment, the invention is directed to a three-elementmonolithic lens optical arrangement for an array camera. In such anembodiment, the optical arrangement includes:

-   -   a first lens element having a first convex proximal surface and        a first plano distal surface;    -   a second lens element having a second concave proximal surface        and a second convex distal surface;    -   a third menisci lens element having a third concave proximal        surface and a third convex distal surface;    -   at least one aperture disposed on the first plano distal        surface; and    -   wherein the first, second and third lens elements are arranged        sequentially in optical alignment with the aperture stop and an        imager.

In another embodiment the three-element monolithic optical arrangementincludes first and second lens elements that are formed from lowdispersion materials and the third lens element is formed from a highdispersion material. In still another such embodiment, at least onefilter is disposed on the first plano distal surface in opticalalignment with the first lens element. In yet another such embodiment,the first lens element further comprises a substrate disposed on thedistal surface thereof. In still yet another such embodiment, at leastone aperture is disposed on the distal surface of said substrate. Instill yet another such embodiment, at least one filter is disposed onthe distal surface of said substrate. In still yet another suchembodiment, the second lens element further comprises a substratedisposed between the proximal and distal surfaces thereof.

In still another embodiment the three-element monolithic lens opticalarrangement is part of an array comprising a plurality of thethree-element monolithic optical arrangements, where the array isdesigned to image a selected wavelength band, and wherein the profile ofat least one of the lens surfaces within each optical arrangement isadapted to optimally image only a narrow-band portion of the selectedwavelength such that in combination the plurality of arrangements withinthe array image the entirety of the selected wavelength band.

In still yet another embodiment, the invention is directed to aplurality of optical arrangements for an array camera including:

-   -   a lens element array stack formed from a plurality of lens        element arrays, each of the lens element arrays being in optical        alignment with each other and a corresponding imager; and    -   where each of the individual lens elements of each of the lens        element stacks is formed from one of either a high or low Abbe        number material, and where the sequence in which one of either a        high or low Abbe number material is used in any individual lens        element stack depends upon the spectral band being detected by        the imager related thereto.

In still yet another embodiment, the invention is directed to aplurality of optical arrangements for an array camera including:

-   -   a lens element array stack formed from a plurality of lens        element arrays, each of said lens element arrays being formed of        a plurality of lens elements;    -   a plurality of structural features integrated into each of the        lens element arrays;    -   where the structural features ensure alignment of the lens        element arrays in relation to each other within the lens element        array stack in at least one dimension.

In one such embodiment of a plurality of optical arrangements thestructural features are selected from the group consisting of lateraland rotational alignment features, spacers and stand-offs.

In still yet another embodiment, the invention is directed to a methodof compensation for systematic fabrication errors in an array having aplurality of optical channels comprising:

-   -   preparing a design incorporating a nominal shape of one of        either a waveplate or a multilevel diffractive phase element        used for only channel-wise color aberration correction of the        optical channels of the array;    -   fabricating an array lens module based on the design;    -   experimentally determining the systematic deviation of the lens        module from the design based on at least one parameter selected        from the group consisting of lens metrologies, centering,        distance and optical performance;    -   redesigning only the channel-wise color aberration correcting        surfaces of the lens module based on the results of the        experiment;    -   refabricating the lens module based on the redesign; and    -   compensating for any of the systematic deviations remaining        using a back focal length of the lens module.

In still yet another embodiment, the invention is directed to an opticalarrangement comprising a plurality of optical channels, each opticalchannel including at least one optical element, comprising at least twooptical surfaces, wherein one of the optical surfaces of each of theplurality of optical channels is a channel specific surface having awavefront deformation sufficient solely to adapt the optical channel toa selected waveband of light. In one such embodiment, the channelspecific surface is selected from the group consisting of waveplates,kinoforms, and radial symmetric multilevel diffractive phase elements.

The features and advantages described in the specification are not allinclusive and, in particular, many additional features and advantageswill be apparent to one of ordinary skill in the art in view of thedrawings, specification, and claims. Moreover, it should be noted thatthe language used in the specification has been principally selected forreadability and instructional purposes, and may not have been selectedto delineate or circumscribe the inventive subject matter.

BRIEF DESCRIPTION OF DRAWINGS

These and other features and advantages of the present invention will bebetter understood by reference to the following detailed descriptionwhen considered in conjunction with the accompanying data and figures,wherein:

FIG. 1 is a plan view of a conventional camera array with a plurality ofimagers.

FIG. 2A is a perspective view of a camera module in accordance withembodiments of the invention.

FIG. 2B is a cross-sectional view of a conventional module in accordancewith embodiments of the invention.

FIG. 3A illustrates a typical plot of the MTF of the optics for a legacycamera cutting-off below the corresponding image sensor's Ny spatialfrequency.

FIG. 3B illustrates a plot of MTF of the optics for a camera arrayhaving bandlimited optical channels at the Nyquist frequency.

FIG. 3C illustrates a plot of MTF of the optics for a camera array inaccordance with embodiments of the invention.

FIG. 4A is a schematic of a three surface two-lens optical arrangementaccording to one embodiment of the invention.

FIG. 4B is a table of exemplary lenses in accordance with one embodimentof the optical arrangement of FIG. 4A.

FIG. 5A1 is a schematic of a five surface three-lens optical arrangementaccording to one embodiment of the invention.

FIG. 5A2 is a table of exemplary lenses in accordance with oneembodiment of the optical arrangement of FIG. 5A1.

FIGS. 5B to 5H are data plots presenting characteristic performanceindicators of the optical arrangement of FIG. 5A1.

FIG. 5I1 is a schematic of a five surface three-lens optical arrangementaccording to one embodiment of the invention.

FIG. 5I2 is a table of exemplary lenses in accordance with oneembodiment of the optical arrangement of FIG. 5I1.

FIG. 5J1 is a schematic of a five surface three-lens optical arrangementaccording to one embodiment of the invention.

FIG. 5J2 is a table of exemplary lenses in accordance with oneembodiment of the optical arrangement of FIG. 5J1.

FIGS. 6A and 6B are schematics of conventional monolithic lens andaperture arrangements.

FIG. 6C is a schematic of a monolithic optical arrangement according toone embodiment of the invention.

FIGS. 6D1 to 6D6 are schematics of monolithic optical arrangementsaccording to various embodiments of the invention.

FIG. 7A is a schematic of a process flow for manufacturing a monolithicoptical arrangement according to one embodiment of the invention.

FIGS. 7B to 7D are schematics of monolithic optical arrangementsaccording to various embodiments of the invention.

FIGS. 7E to 7G are schematics of optical arrangements which incorporatemonolithic lens elements according to the embodiments of the inventionshown in FIGS. 7B to 7D.

FIGS. 8A to 8D are schematics of three element monolithic opticalarrangements according to various embodiments of the invention.

FIGS. 8E to 8J are data graphs of characteristic performance indicatorsof the three-element monolithic optical arrangements according to oneembodiment of the invention.

FIGS. 9A and 9B are schematics of conventional injection molded opticalarrangement formed of two materials.

FIG. 9C is a schematic of an injection molded optical arrangement formedof two materials according to one embodiment of the invention.

FIG. 9D is a schematic of a conventional polymer on glass wafer leveloptical arrangement formed of two materials.

FIG. 9E is a schematic of a polymer on glass wafer level opticalarrangement formed of two materials according to one embodiment of theinvention.

FIG. 10A is a schematic of a conventional polymer on glass wafer leveloptical arrangement having an integrated aperture stop.

FIG. 10B is a schematic of a polymer on glass wafer level opticalarrangement having an integrated aperture stop according to oneembodiment of the invention.

FIGS. 11A and 11B are schematics of optical arrangements havingpreformed spacing and alignment elements according to one embodiment ofthe invention.

FIG. 12 is a flowchart of a process for manufacturing an opticalarrangement according to one embodiment of the invention.

DETAILED DESCRIPTION

Turning now to the drawings, novel optical arrangements for use in anarray camera that captures images using a distributed approach using aplurality of imagers (cameras) of different imaging characteristics areillustrated. In many embodiments, each imager (camera) of such a cameraarray may be combined with separate optics (lens stacks) with differentfilters and operate with different operating parameters (e.g., exposuretime). As will be described, in some embodiments these distinct opticalelements may be fabricated using any suitable technique, including, forexample, injection molding, precision glass molding, polymer-on-glasswafer level optics (WLO), or monolithic-lens WLO technologies (polymeror glass). In other embodiments, the various lens stacks of theindividual cameras and camera array are implemented such that the MTFcharacteristics of the optics include contrast at a spatial frequencythat is at least as large as the resolution of the high resolutionimages to be synthesized by the array camera from the low resolutionimages formed from the individual cameras, and significantly greaterthan the Nyquist frequency of the pixels in the focal plane.

Array cameras including camera modules that can be utilized to captureimage data from different viewpoints (i.e. light field images) aredisclosed in U.S. patent application Ser. No. 12/935,504 entitled“Capturing and Processing of Images using Monolithic Camera Array withHeterogeneous Imagers” to Venkataraman et al. In many instances, fusionand super-resolution processes such as those described in U.S. patentapplication Ser. No. 12/967,807 entitled “Systems and Methods forSynthesizing High Resolution Images Using Super-Resolution Processes” toLelescu et al., can be utilized to synthesize a higher resolution 2Dimage or a stereo pair of higher resolution 2D images from the lowerresolution images in the light field captured by an array camera. Theterms high or higher resolution and low or lower resolution are usedhere in a relative sense and not to indicate the specific resolutions ofthe images captured by the array camera. The disclosures of U.S. patentapplication Ser. No. 12/935,504 and U.S. patent application Ser. No.12/967,807 are hereby incorporated by reference in their entirety.

Each two-dimensional (2D) image in a captured light field is from theviewpoint of one of the cameras in the array camera. Due to thedifferent viewpoint of each of the cameras, parallax results invariations in the position of foreground objects within the images ofthe scene. Processes such as those disclosed in U.S. Provisional PatentApplication No. 61/691,666 entitled “Systems and Methods for ParallaxDetection and Correction in Imaged Captured Using Array Cameras” toVenkataraman et al. can be utilized to provide an accurate account ofthe pixel disparity as a result of parallax between the differentcameras in an array. The disclosure of U.S. Patent Application Ser. No.61/691,666 is hereby incorporated by reference in its entirety. Arraycameras can use disparity between pixels in images within a light fieldto generate a depth map from a reference viewpoint. A depth mapindicates the distance of the surfaces of scene objects from thereference viewpoint and can be utilized to determine scene dependentgeometric corrections to apply to the pixels from each of the imageswithin a captured light field to eliminate disparity when performingfusion and/or super-resolution processing.

The ultimate spatial resolution limit of a camera is inverselyproportional to the pixel size or pitch of the imaging sensor of thecamera and is defined as the Nyquist frequency limit. The Nyquistfrequency states that the maximum resolution, R, of a system is equal tothe inverse of two times the pixel pitch, x, (R=/(2*x)). For a legacycamera very little contrast is desired at spatial frequencies largerthan the sensor's Ny, in order to avoid aliasing in the final outputimage. Accordingly, in a legacy camera the pixel pitch determines thespatial sampling rate, and the corresponding Nyquist frequency (Ny),which is simply one half of the reciprocal of the center-to-center pixelspacing. As such, in a mobile imaging legacy camera, the required MTFfor the optics arrangement is usually specified at Ny/4, Ny/2, or Ny, asillustrated in the plot of FIG. 3A, i.e., to be at least opticallylimited by the pixel pitch.

The challenge in implementing optics for array cameras results from therequirements necessary to achieve the super-resolution processesdescribed above. Of importance is that super resolution should be ableto recover a higher resolution final output image than the intrinsicresolution in the input component images from the individual cameras.Generally this requires that the camera optically resolve, withsufficient contrast, spatial frequencies that are actually larger thanthe Nyquist frequency of the individual sensors. The spatial resolutionof a lens may be specified in terms of the modulation transfer function(MTF) curve over a range of spatial frequencies. As previouslydescribed, the MTF is a spatial frequency response (SFR) of outputsignal contrast with input spatial frequency. Performance is measured interms of contrast or modulation at a particular spatial frequency whichis customarily specified in line pairs per millimeter. At low linefrequencies, the imaging system typically passes the signalunattenuated, which implies a contrast of 100%. At higher linefrequencies, the signal is attenuated and the degree of attenuation inthe output signal is expressed as a percentage with respect to that ofthe input signal, normalized to unity (or 100%) contrast at zero spatialfrequency. In other words, the MTF is a measure of the ability of anoptical system to transfer various levels of detail from object toimage. Accordingly, embodiments of array cameras have more stringent MTFrequirements than for legacy cameras, and in particular use opticshaving MTF characteristics that exceed the spatial resolution (Nyquistfrequency) of the pixel pitch of the pixels on a focal plane.

In particular, when multiple copies of an aliased signal are present, asin embodiments of a camera array, it is possible to use the informationthat is inherently present in the aliasing to reconstruct a higherresolution signal. However, there are slight differences between thealiasing patterns in the different camera images due to the array'ssampling diversity. This sampling diversity is the result of slightlydifferent viewing directions of the different cameras within the array,which are either intentionally introduced or result from (positional)manufacturing tolerances. Typically, filters would be introduced tocreate a bandlimited signal having an MTF near the Ny of the sensor asillustrated in FIG. 3B. However, in embodiments of a camera arrayaliasing to create a super resolution image is necessary. Accordingly,to provide sufficient contrast in the aliased LR images, the lens MTFneeds to be as high as the desired high resolution output MTF from thesuper resolution processing. As illustrated in FIG. 3C, this requiresthat the cameras capture content above Ny such that the super-resolutionprocess can then recover the higher resolution information. To addressthis optics challenge, in many embodiments, the MTF characteristics ofthe optics in camera arrays are implemented such that images formedinclude contrast at a spatial frequency that is at least as great as theresolution of the high resolution images synthesized by the arraycamera, and significantly greater than the Nyquist frequency of thepixel pitch of the pixels on the focal plane (e.g., in some embodimentsfrom 1.5 to 3 times Ny). In many embodiments, the specific MTFrequirement of the optics of the camera array may be determined by theratio of the resolution of the high resolution image and Nyquistresolution of the individual camera. In other words, in some embodimentsoptics lens are implemented having a contrast of at least 10%, in someembodiments at least 20%, and in other embodiments at least 30% at aspatial frequency given by the ratio of the number of line-pairsresolved in the final synthesized high resolution image, and thephysical size of the low resolution camera image in the same dimension,where the physical size of the low resolution image is a function of thesize and number of pixels in the individual camera along the relevantdimension.

DEFINITIONS

A sensor element or pixel refers to an individual light-sensing elementin a camera array. The sensor element or pixel includes, among others,traditional CIS (CMOS Image Sensor), CCD (charge-coupled device),quantum dot films, high dynamic range pixel, multispectral pixel andvarious alternatives thereof. The pixel pitch of these sensor elementsdefines the Nyquist frequency.

An imager refers to a focal plane formed from a two dimensional array ofpixels associate with a lens stack formed from a set of opticalelements. The sensor elements or pixels of each imager or focal planehave similar physical properties and receive light through the same setof optical components or lens stack. Further, the sensor elements in theeach imager/focal plane may be associated with the same color filter.

An imager or camera array refers to a collection of imagers/camerasdesigned to function as a unitary component. The imager or camera arraymay be fabricated on a single chip for mounting or installing in variousdevices.

A lens stack refers to an axial arrangement of several opticalcomponents/lens elements.

An optical channel refers to the combination of a lens stack and animager or focal plane.

A lens or optical array refers to a lateral arrangement of individuallens elements stacks.

An optics or lens stack array refers to a lateral array of lens stacks,or an axial arrangement of multiple lens arrays.

A camera array module refers to the combination of an optics array andan imager array, and can also be defined as an array of opticalchannels.

Image characteristics of an imager refer to any characteristics orparameters of the imager associated with capturing of images. Theimaging characteristics may include, among others, the size of theimager, the type of pixels included in the imager, the shape of theimager, filters associated with the imager, the exposure time of theimager, aperture size of the optics associated with the imager, theconfiguration of the optical element associated with the imager, gain ofthe imager, the resolution of the imager, and operational timing of theimager. The characteristics of the optics of a camera refer to at leastthe field of view (FOV), F-number (F/#), resolution (MTF), effectivefocal length or magnification, color or waveband, distortion, andrelative illumination.

These defined aspects of the embodiments will be described in greaterdetail below.

Structure of Array Camera

Array cameras in accordance with embodiments of the invention caninclude a camera module and a processor. FIG. 1 is a plan view of ageneric array camera 100, which includes a camera module (110 with anarray of cameras orimagers 1A through NM. As shown, a camera module ofthe type shown is fabricated to include a plurality or array of cameras1A through NM. In turn, each of the cameras 1A through NM may include aplurality of focal planes and light sensing pixels (e.g., 0.32 Megapixels). Although the imagers 1A through NM are shown as arranged into agrid format, it should be understood that they may be arranged in anysuitable configuration. For example, in other embodiments, the imagersmay be arranged in a non-grid format, such as in a circular pattern,zigzagged pattern or scattered pattern.

These array cameras may be designed as a drop-in replacement forexisting camera image sensors used in cell phones and other mobiledevices. For this purpose, the camera array may be designed to bephysically compatible with conventional camera modules of approximatelythe same resolution although the achieved resolution of the camera arraymay exceed conventional image sensors in many photographic situations.Taking advantage of the increased performance, the array camera of theembodiment may include an imager with fewer pixels to obtain equal orbetter quality images compared to conventional image sensors.Alternatively, the size of the pixels in the imager may be reducedcompared to pixels in conventional image sensors while achievingcomparable results. In some embodiments, the array camera replaces aconventional image sensor of M megapixels. The array camera has (N×N)individual imagers or cameras, each camera including pixels of M/N².Each camera in the camera array may also have the same aspect ratio asthe conventional image sensor being replaced.

Array Camera Modules

Camera modules in accordance with embodiments of the invention can beconstructed from an imager array and an optic array. Camera modules inaccordance with embodiments of the invention are illustrated in FIGS. 2Aand 2B. The camera module 200 includes an imager array 230 including anarray of focal planes 240 along with a corresponding optic array 210including an array of lens stacks 220. Within the array of lens stacks,each lens stack 220 creates an optical channel that forms an image ofthe scene on an array of light sensitive pixels 242 within acorresponding focal plane 240. As is described further below, the lightsensitive pixels 242 can be formed from quantum films. Each pairing of alens stack 220 and focal plane 240 forms a single camera 104 within thecamera module. Each pixel within a focal plane 240 of a camera 104generates image data that can be sent from the camera 104 to theprocessor 108. In many embodiments, the lens stack within each opticalchannel is configured so that pixels of each focal plane 240 sample thesame object space or region within the scene. In several embodiments,the lens stacks are configured so that the pixels that sample the sameobject space do so with sub-pixel offsets to provide sampling diversitythat can be utilized to recover increased resolution through the use ofsuper-resolution processes. The camera module may be fabricated on asingle chip for mounting or installing in various devices.

In several embodiments, an array camera generates image data frommultiple focal planes and uses a processor to synthesize one or moreimages of a scene. In certain embodiments, the image data captured by asingle focal plane in the sensor array can constitute a low resolutionimage (the term low resolution here is used only to contrast with higherresolution images), which the processor can use in combination withother low resolution image data captured by the camera module toconstruct a higher resolution image through Super Resolution processing,as previously described. Where super resolution is performed thenmultiple copies of an aliased signal are present, such as in multipleimages from the focal planes 240, and the information inherently presentin the aliasing may be used to reconstruct the higher resolution signal.One skilled in the art will note that the aliasing patterns from thedifferent focal planes 240 will have slight differences due to thesampling diversity of the focal planes. These slight differences resultfrom the slightly different viewing directions of the cameras used tocapture the low resolution images that are either intentionallyintroduced or result from positional manufacturing tolerances of theindividual focal planes. Thus, in accordance with some embodiments ofthis invention, the MTFs of the lens stacks 220 need to be at least ashigh as the desired high resolution output MTF to provide sufficientcontrast. Accordingly, in many embodiments of an array camera, optics inthe lens stack are implemented that have an MTF at least as high as thedesired MTF of the super resolution image, i.e., an MTF at which theindividual optic channels of the camera array are capable of resolvingspatial frequencies above the Nyquist frequency of the pixels on thefocal plane. In many such embodiments, the optics have an MTF at whichthey are capable of resolving spatial frequencies at least 1.5, 2 and/or3 times the Nyquist frequency of the pixels to allow thesuper-resolution process to recover higher resolution informationunavailable from the individual low resolution images captured at theindividual cameras.

Imager Arrays

Imager arrays in accordance with embodiments of the invention can beconstructed from an array of focal planes formed of arrays of lightsensitive pixels. As discussed above in relation to FIG. 2A, in manyembodiments the imager array 230 is composed of multiple focal planes240, each of which have a corresponding lens stack 220 that directslight from the scene through optical channel and onto a plurality oflight sensing elements (the pixel pitch of which define Ny) formed onthe focal plane 240. In many embodiments the light sensing elements areformed on a CMOS device using photodiodes formed in the silicon wherethe depleted areas used for photon to electron conversion are disposedat specific depths within the bulk of the silicon. In some embodiments,a focal plane of an array of light sensitive pixels formed from aquantum film sensor may be implemented. The formation, composition,performance and function of various quantum films, and their use inoptical detection in association with semiconductor integrated circuitsare described in U.S. Patent Publication US/2009/0152664, entitled“Materials, Systems and Methods for Optoelectronic Devices”, publishedJun. 18, 2009, the disclosure of which is incorporated by referenceherein in its entirety.

A focal plane 240 in accordance with an embodiment of the inventionincludes a focal plane array core that includes an array of lightsensitive pixels 242 disposed at the focal plane of the lens stack 220of a camera on a semiconducting integrated circuit substrate 230, suchas a CMOS or CCD. The focal plane can also include all analog signalprocessing, pixel level control logic, signaling, and analog-to-digitalconversion (ADC) circuitry used in the readout of pixels. The lens stack220 of the camera directs light from the scene and onto the lightsensitive pixels 242. The formation, architecture and operation ofimager arrays and light sensitive pixel arrays, and their use in opticaldetection in association with array cameras are described in U.S. patentapplication Ser. No. 13/106,797, entitled “Architectures for ImagerArrays and Array Cameras”, filed May 12, 2011, the disclosure of whichis incorporated by reference herein in its entirety.

Imager arrays of this design may include two or more types ofheterogeneous imagers, each imager or camera including two or moresensor elements or pixels. Each one of the imagers may have differentimaging characteristics. Alternatively, there may be two or moredifferent types of imagers where the same type of imagers shares thesame imaging characteristics. For example, each imager 1A through NM inFIG. 1 may be associated with its own filter and/or optical element(e.g., lens). Specifically, each of the imagers 1A through NM or a groupof imagers may be associated with spectral color filters to receivecertain wavelengths of light. Example filters include a traditionalfilter used in the Bayer pattern (R, G, B), an IR-cut filter, a near-IRfilter, a polarizing filter, and a custom filter to suit the needs ofhyper-spectral imaging. In addition, some imagers may have no filter toallow reception of both the entire visible spectra and near-IR, whichincreases the imager's signal-to-noise ratio. The number of distinctfilters may be as large as the number of cameras in the camera array.Embodiments where filter groups are formed is further discussed in U.S.Provisional Patent Application No. 61/641,165 entitled “Camera ModulesPatterned with pi Filter Groups” filed May 1, 2012, the disclosure ofwhich is incorporated by reference herein in its entirety. These camerascan be used to capture data with respect to different colors, or aspecific portion of the spectrum. In other words, instead of applyingcolor filters at the pixel level of the camera, color filters in manyembodiments of the invention are included in the lens stack of thecamera. For example, a green color camera can include a lens stack witha green light filter that allows green light to pass through the opticalchannel. In many embodiments, the pixels in each focal plane are thesame and the light information captured by the pixels is differentiatedby the color filters in the corresponding lens stack for each filter.

It will be understood that such imager arrays may include other relatedcircuitry. The other circuitry may include, among others, circuitry tocontrol imaging parameters and sensors to sense physical parameters. Thecontrol circuitry may control imaging parameters such as exposure times,gain, and black level offset. The sensor may include dark pixels toestimate dark current at the operating temperature. The dark current maybe measured for on-the-fly compensation for any thermal creep that thesubstrate may suffer from.

Optic Arrays

To provide lenses and other optical elements for implementation in thelens stacks of the optical arrays any suitable optical technology may beemployed capable of forming a lens stack with a suitable MTF, i.e., anoptic MTF at least as high as the MTF of the high resolution image to beobtained via super resolution. To determine the suitability of opticalelements and lens stacks, each lens stack 220 may be specified in termsof its MTF curve over a range of spatial frequencies. As the MTF is aSpatial Frequency Response (SFR) of the output signal contrast with theinput spatial frequency, it is possible to determine the frequenciesthat can be optically resolved with sufficient frequency by theindividual optical elements and lens stacks.

One skilled in the art will understand that any lens system willdemonstrate a number of MTF curves depending on the operating conditionsof the camera (such as aperture size) and the spatial frequency beingresolved. The MTF might also be impacted by the type of scene beingimaged (for example, there is often a spread in the MTF curve between alenses ability to resolve meridional and sagittal lines). Finally, alens system may demonstrate various levels of spatial resolution as youproceed from the center of image outward. In many embodiments of anarray camera, optics are selected such that the individual cameras ofthe camera array are able to spatially resolve at frequencies above theNyquist frequency of the pixels across all imaging conditions andlocations on the lens to allow the super-resolution process to recoverhigher resolution information under all conditions and camera settings.In other embodiments, however, lens stacks and optical elements arecontemplated that demonstrate an MTF sufficiently high to allow for thecamera to spatially resolve at frequencies above the Nyquist frequencyof the pixels across only some imaging conditions and camera settings. Adetailed description of various optical elements for use in cameraarrays is provided below.

FIG. 2A illustrates a perspective view of one array camera assembly 200that incorporates an optics array 210 with an imager array 230. Asshown, the optics array 210 generally includes a plurality of lensstacks 220 (which furthermore may consist of several axially alignedlens elements), each lens stack 220 covering (in the shown example) oneof twenty-five imagers 240 in the imager array 230.

FIG. 2B illustrates a sectional view of a camera array assembly 250. Asshown, in such a design the camera assembly 250 would comprise an opticsarray including a top lens wafer 262 and a bottom lens wafer 268, and animager array including a substrate 278 with multiple sensors andassociated light sensing elements formed thereon. Spacers 258, 264 and270 are also included to provide proper positioning to the variouselements. In this embodiment the camera array assembly 250 is alsopackaged within an encapsulation 254. Finally, an optional top spacer258 may be placed between the encapsulation 254 and the top lens wafer262 of the imager array; however, it is not essential to theconstruction of the camera assembly 250. Within the imager array,individual lens elements 288 are formed on the top lens wafer 262.Although these lens elements 288 are shown as being identical in FIG.2B, it should be understood that within the same camera array differenttypes, sizes, and shapes of elements may be used. Another set of lenselements 286 is formed on the bottom lens wafer 268. The combination ofthe lens elements on the top lens wafer and bottom lens wafer form thelens stacks 220 shown in FIG. 2A.

In these types of camera arrays, through-silicon vias 274 may also beprovided to paths for transmitting signal from the imagers. The top lenswafer 262 may be partially coated with light blocking materials 284(e.g., chromium, oxidized (“black”) chromium, opaque photoresist) toblock of light. In such embodiment, the portions of the top lens wafer262 of the optics array not coated with the blocking materials 284 serveas apertures through which light passes to the bottom lens wafer 268 andthe imager array. Although only a single aperture is shown in theembodiment provided in FIG. 2B, it should be understood that, in thesetypes of camera arrays, additional apertures may be formed from opaquelayers disposed on any and all of the substrate faces in the cameraassembly to improve stray light performance and reduced opticalcrosstalk. In the example shown in FIG. 2B, filters 282 are formed onthe bottom lens wafer 268 of the optics array. Light blocking materials280 may also be coated on the bottom lens wafer 268 of the optics arrayto function as an optical isolator. A light blocking material 280 mayalso be coated on the substrate 278 of the imager array to protect thesensor electronics from incident radiation. Spacers 283 can also beplaced between the bottom lens wafer 268 of the optics array and thesubstrate 278 of the imager array, and between the lens wafers 262 and268 of the optics array. In such array cameras, each layer of spacersmay be implemented using a single plate.

Although not illustrated in FIG. 2B, many such camera arrays alsoinclude spacers between each optical channel located on top of the toplens wafer 262 of the optics array that are similar to, or implementedin single layer with, the spacer 258 shown at the edge of the lens stackarray. As is discussed further below the spacers can be constructed fromand/or coated in light blocking materials to isolate the opticalchannels formed by the wafer level optics. Suitable light blockingmaterials may include any opaque material, such as, for example, a metalmaterial like Ti and Cr, or an oxide of these materials like blackchromium (chrome and chrome oxide), or dark silicon, or a black particlefilled photoresist like a black matrix polymer (PSK2000 from BrewerScience).

There are a number of advantages that can be realized by using smallerlens elements with these array cameras. First, the smaller lens elementsoccupy much less space compared to a single large lens covering theentire camera array 230. In addition, some of the natural consequencesof using these smaller lens elements include, improved opticalproperties by reduced aberrations, in particular chromatic aberrations,reduced cost, reduced amount of materials needed, and the reduction inthe manufacturing steps. A full discussion of these advantages may befound in U.S. Patent Publication No. US-2011-0080487-A1, the disclosureof which is incorporated herein by reference.

Because of the distributed approach of the array camera, and theresultant relaxed total track length requirement (since the array cameraby its nature is much shorter than a comparable classical objective), itis possible to adopt novel optical design approaches for the lenschannels of an array camera rather than just using arrays ofconventional optical designs. In particular, many embodiments aredirected to optic arrangements capable of MTF characteristics such thatimages formed on a focal plane include contrast at a spatial frequencythat is at least greater than the resolution of high resolution imagessynthesized by the array camera during super resolution, andsignificantly greater than the Nyquist frequency of the pixel pitch ofthe pixels on the focal plane, and in some cases as much as 1.5 to 3times the Nyquist frequency. These novel arrangements will be describedin detail below, however, it should be understood that other opticalarrangements that incorporate the improvements set forth herein may beused with the camera arrays described herein.

Embodiment 1 Three-Surface WLO Design

Traditional wafer level optics (WLO) is a technology where polymerlenses are molded on glass wafers, potentially on both sides, stackedwith further such lens wafers by spacer wafers, and diced into lensmodules (this is called “polymer on glass WLO”) followed by packaging ofthe optics directly with the imager into a monolithic integrated module.As will be described in greater detail below, the WLO procedure mayinvolve, among other procedures, using a wafer level mold to create thepolymer lens elements on a glass substrate. Usually this involvesincorporating apertures, and in particular the aperture stop byproviding openings centered with the later lens channels in an otherwiseopaque layer onto the substrate before lens molding.

In a first embodiment, a three-surface optical arrangement suitable forthe fabrication by wafer level optics technology, and, in particular, tobe used for the optics (as one of the multiple channels) of an arraycamera is described in reference to FIG. 4A. More specifically, in astandard two-element lens, as shown in FIGS. 2 and 3, there aretypically four lens surfaces (front and back for the top and the bottomlenses). In contrast, in this total-track-length-relaxed butMTF-performance optimized design, the third surface (first side ofsecond element) has very low refractive power. As a result, it ispossible to omit it from the design entirely. The result is a threesurface design, which leads to a less expensive lens, due to lessrequired process steps, and improved yield because of less contributorsto the yield multiplication. In addition, since only lenses that haveshapes which appear close to spherical- or parabolic profiles(monotonous profiles, no wings) are applied in a specific axialarrangement, centered around the aperture stop, only weak ray bendingoccurs at all the refractions on air-lens or lens-air interfaces. Theresult of this arrangement is a relaxed sensitivity with respect tocentering-, thickness-, distance-, or lens shape form error tolerances.As shown in FIG. 4A, the rays for the different field heights more orless transmit perpendicularly, and are thus not strongly refractedthrough the lens surfaces. However, in such an arrangement it is veryimportant to find an optimum position where the angle of incidence (AOI)on the glass substrate is minimal so that the shift of the band edgesdue to the AOI is minimized for any dielectric filter system (e.g. forIR cut-off), which is applied on substrates within the lens stack.

As shown FIG. 4A, the three-surface optical arrangement is identified byfirst 400 and second lens elements 402, which are arranged sequentiallyalong a single optical path 403. It should be understood that forconstruction purposes, each lens element may optionally be associatedwith a corresponding supporting substrate 404 and 406, made from exampleof glass, upon which the polymer lens surfaces are formed. In addition,spacer elements (not shown) that can serve to mechanically connect thelens elements to each other and/or to the image sensor may also beincluded in the construction. Although any suitable material may beused, in one embodiment the lens surfaces are made from a (UV- orthermally curable) polymer.

Turning to the construction of the lens elements themselves, in thefirst lens element 400, there is a convex surface 408 of a firstdiameter on the first side of the first element and a concave surface410 of a second diameter on the second side of the first element.Preferably the diameter of the first side is larger than the diameter ofthe second side of the first lens element. In the second element 402,there is a shallow or flat surface 412 on the first side of the secondelement, and a convex surface 414 on the second side of the secondelement. Preferably, the diameter of the first side of the secondelement is smaller than the diameter of the second surface of the secondelement, and that the diameter of the second side of the second elementis intermediate between the diameters of the first and second sides ofthe first element. In addition, the system aperture or stop (not shown)is preferably disposed on the first side of the second element.

Although not shown in the diagram, a (thin) first spacing structure (notshown) is placed in between the two lens elements, which can be eitherincorporated into the respective lens surfaces (“stand-offs”), or can bean additional element. Likewise, a (thick) second spacing structureconnecting the second side of the second lens element with the coverglass or package 416 of the image sensor 417 may also be provided. Bothspacing structures are preferably opaque, or have opaque surfaces, atleast at the inner side-walls, and provide partial optical isolationbetween adjacent optical channels. FIG. 4B provides a lens table for anexemplary embodiment of the three-surface optical arrangement inaccordance with the current invention.

Although the basic construction of the three-side optical arrangement isdescribed above, it should be understood that other features andstructures may be incorporated into the lens elements to provideadditional or enhanced functionality, including:

-   -   The inclusion of additional apertures within the lens stack (in        particular on the glass substrates underneath the polymer        lenses).    -   Channel specific filters, such as, for example, organic color        filter arrays “CFA” and/or structured dielectric filters, such        as, for example, IR cut-off, NIR-pass, interference color        filters. These filters may be arranged within the stack of the        first and the second lens element, preferably in a surface close        to the system aperture.    -   Partial achromatization of the individual        narrow-spectral-band-channels may be accomplished by combining        different Abbe-number materials for the different lens surfaces,        such as, for example, “crown-like” materials for the two convex        surfaces on the outsides of the optical arrangement, and        “flint-like” materials for the potentially two (concave)        surfaces on the inner sides of the two lens elements (as is        further described in Embodiment 6, below).    -   Optimization of different color channels to their specific        narrow spectral band may be accomplished by adapting (at least)        one lens surface profile within the optical arrangement to that        color to correct for chromatic aberrations. (For a full        discussion see, e.g., U.S. patent application Ser. No.        13/050,429, the disclosure of which is incorporated herein by        reference.)

There are several features of this novel three-surface opticalarrangement that render it particularly suitable for use in arraycameras. First, the optical arrangement is designed in such a way thatvery high contrast at the image sensor's Nyquist spatial frequency isachieved, which at the same time (for gradual fall-off of contrast withincreasing spatial frequency) provides sufficient contrast at 1.5× or 2×the sensor's Nyquist frequency to allow the super-resolution imageinformation recovery to work effectively. Second, the opticalarrangement is optimized for allowing a small lateral distance betweenadjacent optical channels in order to economically exploit the diereal-estate area, consequently the lens diameters should be small, asshould the wall-thickness of the (opaque) spacer structures. Third,optical channels within one array dedicated to imaging different“colors” (parts of the overall wavelength spectrum to be captured) maydiffer with regard to the particular surface profile of at least onelens surface. The differences in the surface profiles of those lenses inone array can be minor, but are very effective in keeping the back focallength (“BFL”) color-independent, and consequently allowing (almost)equally sharp images for the different colors without the costly needfor wide-spectral-band achromatization. Moreover, after computationalcolor-fusion a high-resolution polychromatic image can be achieved. Inparticular, preferably the last surface in the lens stack (here—secondsurface of second element) is specifically optimized for the narrowspectral band of the respective color channel. Fourth, the above designapproaches result in a (partial—as far it can be the case for the givennon-symmetry between object- and image space) symmetry of the lenssystem around the aperture stop, which helps to reduce certain types ofaberrations, including, distortion, coma and lateral color.

The benefits of this array-dedicated design of the single channels ofthe array camera include:

-   -   The ability to provide high resolution with as few as two        elements, and with only three surfaces of refractive power        within the two elements,    -   Increased simplicity of the lens shapes,    -   Reduced fabrication tolerance sensitivity due to reduced ray        bending.    -   Low CRA, due to relaxed total track length requirement.    -   Low color cross-talk, due to color filters differentiating the        different optical channels rather than having a Bayer pattern on        the pixel level performing this task.    -   Low pixel cross-talk, due to the smaller pixel stack height.    -   Reduced color inhomogeneity, due to the color filters being        farther from the image plane.    -   Low inter-channel cross-talk, because of the long back focal        length, which allows a thick opaque second spacer compared to        thin transparent substrate.    -   Fewer monochromatic aberrations, as a result of the smaller        lenses, because many of these aberrations scale with lens size        ̂2-̂4.    -   Separate color channels that only need to be optimized for their        respective spectral bands, resulting in higher overall        polychromatic resolution while minimizing the achromatization        requirements within the individual channels, again resulting in        simpler overall aberration balancing or correction processes and        simpler lenses, and/or better MTF, and/or lower F/#. In many        embodiments the MTF characteristics of the three-surface optical        arrangement allow for contrast at spatial frequencies that are        at least as great as the resolution of the high resolution        images synthesized by the array camera, and significantly        greater than the Nyquist frequency of the pixel pitch of the        pixels on the focal plane, which in some embodiments may be 1.5,        2 or 3 times the Nyquist frequency.    -   Higher yield during manufacture, because the smaller lenses mean        smaller sag (i.e., vertex height) of the lenses, which leads to        less shrinkage and the ability to use less complex replication        technology.

Embodiment 2 Five-Surface WLO Design

In a second embodiment, a five-surface optical arrangement suitable forthe fabrication by wafer level optics technology, and, in particular, tobe used for the optics (as one of the multiple channels) of an arraycamera is described in reference to FIGS. 5A to 5J. More specifically,this embodiment is directed to a five-surface high-resolution waferlevel lens/objective having a field-flattening element close to theimage plane. Again, in a standard two-element lens, as shown in FIGS. 2and 3, there are typically four lens surfaces (front and back for thetop and the bottom lenses). However, these optical arrangements arenon-ideal for use in high resolution array cameras. Ideally, there wouldbe a reduced requirement on small maximum CRA of the optics by the imagesensor (allowing much larger angles of incidence on the same), forexample, by using BSI or quantum film sensors (quantum film having theadditional advantage of an increased fill factor not requiringmicrolenses, which otherwise require an air gap in front of the imagesensor in order for the microlenses to provide refractive power).Finally, regular lens designs have stronger small total track lengthrequirements, which would render the overall camera length shorter thannecessary for an array camera, since the array camera is much shorterthan a comparable classical objective by concept. In contrast, in thisdesign five surfaces are used. The large number of degrees of freedom inthe five-surface design allows for achromatization for the full visiblespectral band (or other band of interest), so that channel-specific lensprofiles are not necessarily required. However, even though the backfocal length may be kept constant over the spectral band of interest,the effective focal length and with it the magnification may vary.

As shown in FIG. 5A, in one embodiment the five-surface opticalarrangement is identified by first 500, second 502, and third 504 lenselements arranged sequentially along a single optical path 505. Itshould be understood that for construction purposes, each lens elementmay optionally be associated with a corresponding supporting substrate506, 508 & 510, made from example of glass, upon which the polymer lenssurfaces are formed. In addition, spacer elements (not shown) that canserve to mechanically connects the lens elements to each other and/or tothe image sensor may also be included in the construction. Although anysuitable material may be used, in one embodiment the lens surfaces,i.e., the volume between the surface of the lens and the underlyingsubstrate, are made from a (UV- or thermally curable) polymer.

Turning to the construction of the lens elements themselves, the firstlens element 500 has a convex surface 512 having a first diameter, and aconcave surface 514 having a second diameter. Preferably the diameter ofthe convex surface is greater than the diameter of the concave surfaceon this lens element. The second lens element 502 has a concave surface516 on the first side of the second lens element, and a convex surface518 on the second side of the second element. In this second lenselement, preferably the convex surface thereof is of a larger diameterwhen compared to the concave surface thereof. The third lens element 504has a concave surface 520 on the first side of the third lens element,and a second planar side 522 that is adjoined to the substrate thatserves as the image sensor cover 510. Typically, the diameter of theconcave surface of the third lens element is larger than the diametersof any of the surfaces of the first and second lens elements.

In terms of arrangement, a first spacing structure (not shown) isdisposed in between the first 500 and the second 502 lens elements.Likewise, a second spacing structure (not shown) is disposed in betweenthe second 502 and the third 504 lens elements. Either of these spacersmay be either incorporated (also split) into the respective lenssurfaces (“stand-offs” in which the lenses then can be glued directlytogether), or can be an additional element. In addition, both of thesespacing structures are preferably opaque (or have opaque surfaces, atleast at the (inner) side-walls), and provide (partial) opticalisolation between adjacent optical channels. The third lens element 504is disposed comparatively close to the image surface 524, and the secondside of the third lens element is preferably connected with the imagesensor or image sensor cover glass by a transparent areal bond or alocal bond (e.g. UV- and/or thermally curing adhesive), or even a (−nopaque) spacing structure with transparent openings as described above.

In summary, FIG. 5A1, discussed above, demonstrates a five-surfaceoptical arrangement disposed on a large-sag field flattener on aregular-thickness image sensor cover glass. In particular, there is noair gap between the field flattener 504 (or its substrate, or the sensorcover glass, respectively) and the image sensor 524. In front of thefield flattener is the actual focusing objective comprised of first 500and second 502 lens elements, ideally containing two concave surfacesclose to the system aperture more or less symmetrically surrounded bytwo convex surfaces. An exemplary lens table associated with this designis provided in FIG. 5A2.

FIGS. 5B-H present some characteristic performance indicators of thefive-surface optical arrangement shown in FIG. 5A1. In particular, FIG.5B provides a data graph of the Strehl ratio showing that the lens isdiffraction limited over the full field height (@ F/2.4 and diagonalfull FOV of 56°). FIG. 5C provides a data graph showing that in acomparison of MTF vs. field there is virtually no loss of performancewith increasing field height. FIG. 5D provides a data graph of thepolychromatic diffraction encircled energy, and demonstrates that mostof the focused light energy is within the Airy disk. FIG. 5E provides aspot diagram demonstrating that the lens almost appears to beisoplanatic where there is little change of spot size and shape withfield height. FIG. 5F provides a data graph of MTF vs. spatialfrequency, and shows that for small and intermediate field heights thereis still 15-20% contrast even at 500 LP/mm. FIG. 5G provides data graphsshowing that the lens design demonstrates acceptable and monotonousdistortion. Finally, FIG. 5H provides a relative illumination plot,demonstrating that the optical arrangement shows the usual vignettingbehavior.

These data results show that for this particular design family, due tothe strong degree of achromatization, there is very little performanceloss using the green channel for red and blue spectra. In other words,the system is already well achromatized for the full visible spectrum.Accordingly, there is not much benefit when optimizing the green and redchannels specifically rather than just using the green one. Explicitfull visible optimization is very promising as well, i.e., nodifferences between the channels are required. Furthermore, from theseresults it is possible to recognize that optimizing a lens for the fullvisible spectrum, but using only the red, green and blue bandsseparately will improve performance even beyond what is seen from thevisible-polychromatic MTF, Strehl-ratio and Encircled energy plots. Thisis because fewer wavelengths contribute to each color channel'spolychromatic blur. Moreover, this effect becomes more significant themore lateral color dominates the polychromatic blur over axial color,since then the difference between colors is mostly reflected in adifference of magnification or focal length as described above ratherthan different blur sizes. In short, the largest benefit of thesefeatures is that all channels could be the same, simplifying the arraymastering considerably.

Although the above discussion has focused on an embodiment of afive-surface optical arrangement with no air gap between the fieldflattener and the imager, it should be understood that alternativeembodiments incorporating air gaps may be made in accordance with thisinvention. These embodiments are advantageous because they may becombined with regular image sensors which can have fill factor enhancingmicrolenses and a limited maximum CRA of around 30°. For Example, FIGS.5I1 and J1 provide diagrams of two such embodiments, which will bedescribed below.

The embodiment shown in FIG. 5I1 is a five-surface optical arrangementthat allows for an air gap 526 between the sensor cover glass 510 (onwhich the field flattener 528 is positioned) and the image sensor 524.This is usually required when fill factor enhancing plenses are appliedon top of the image sensor 524. As a result of the presence of the airgap, the chief ray angle needs to be reduced over the embodiment shownin FIG. 5A1. Although there are no constraints made on lens vertexheights and minimum glass thicknesses, lens TTL increases and imageperformance reduces due to the requirement of a reduced maximum CRA.However, ray calculations indicate that even in this embodiment the CRAof the inventive optical arrangement meets regular sensor specifications(in the order of magnitude of 27-28° in air). An exemplary lens tableassociated with this design is provided in FIG. 512.

FIG. 5J1 provides a schematic of an embodiment of a five-surface opticalarrangement optimized for best possible manufacturability. Inparticular, in this embodiment the lens sags are decreased, and lensmaterial planar base layers 532, 534, 536, 538 and 542 having suitablethicknesses are provided. It should be understood that for constructionpurposes, each lens element may optionally be associated with acorresponding supporting substrate 533 & 533′, 537 & 543 made fromexample of glass, upon which the polymer lens surfaces and base layersmay be formed. In addition, spacer elements (not shown) that can serveto mechanically connect the lens elements to each other and/or to theimage sensor may also be included in the construction.

In addition, the system aperture 540 is sandwiched (or “embedded”)between two glass substrates 533 and 533′ in order to decrease thenecessary polymer thickness of the adjacent lens surfaces. Finally, aglass substrate 543 is provided between the field flattener lens surface544 and the imager package, including the image sensor glass cover 545and the image sensor 546 (with air gap 548). Although an even split of50/50 is shown in FIG. 5J1, the thickness between the glass substrate543 and the image sensor cover glass 545 may be shared by any reasonableratio (which allows sufficient thickness for both). Cover glass asneeded for the imager may also be provided. All of these elements maythen be immersed and bonded together by a suitable adhesive duringmanufacture. Again, in this embodiment the CRA meets regular sensorspecifications (in the order of magnitude of 27-28° in air). Anexemplary lens table associated with this design is provided in FIG.5J2.

Although the basic construction of the five-surface optical arrangementis described above, it should be understood that other features andstructures may be incorporated into the lens elements to provideadditional or enhanced functionality (references are to FIG. 5A1),including:

-   -   The system aperture (Stop) may be disposed either on the second        side of the first element 500, or on the corresponding side of        the respective glass substrate, or embedded within the first        lens element 500 (e.g. sandwiched between two thinner glass        substrates that have been structured with an aperture array on        the inner side, then glued together). In such an embodiment, the        lenses of the element would be replicated on this aperture        sandwich.    -   As discussed above, other implementations of this general design        may have either no air gap or a thin air gap between the third        lens element 504 and the photosensitive surface (or interface        thereto) of the image sensor 524. Such a design allows this        optical arrangement to operate with regular image sensors,        specifically those with fill factor-enhancing microlenses, and        image sensors with conventional CRA. However, this results in        longer TTL and only moderate image quality compared to versions        without the discussed air gap. In particular, the CRA needs to        be moderate when there is an air gap at this location, because        otherwise there can be (partly) total internal reflection at the        interface between the higher refractive index third lens element        504 and the air gap, or such a strong refraction outwards that        strong aberrations occur, i.e., rays may be fanned out rather        than focused.    -   Several additional apertures may also be disposed within the        stack, and, in particular, on the glass substrates underneath        the polymer lenses where applicable.    -   Channel specific filters may also be arranged within the stack        of the first 500 and the second 502 lens element, preferably in        a surface close to the system aperture. Such filters may        include, for example, organic color filter array “CFA” and/or        structured dielectric filters, such as, IR cut-off or NIR-pass        interference color filters.    -   Partial achromatization of the individual        narrow-spectral-band-channels may be accomplished by combining        different Abbe-number materials for the different lens surfaces.        Preferably “crown-like” materials would be used for the two        convex surfaces on the outsides of the two first lens elements        and “flint-like” materials for the two concave surfaces on the        inner sides of the two first lens elements (See Embodiment 6).    -   Optimization of the different color channels to their specific        narrow spectral band may also be accomplished by adapting at        least one lens surface profile within the optical arrangement to        that color to correct for chromatic aberrations. (For a full        discussion see, e.g., U.S. patent application Ser. No.        13/050,429, the disclosure of which is incorporated herein by        reference.)

There are several features of this novel five-surface opticalarrangement that render it particularly suitable for use in arraycameras. First, the optical arrangement is designed in such a way thatvery high contrast at the used image sensor's Nyquist spatial frequencyis achieved, which at the same time (for gradual fall-off of contrastwith increasing spatial frequency) provides sufficient contrast at 1.5×or 2× the sensor's Nyquist frequency to allow the super-resolution imageinformation recovery to work effectively. Second, the opticalarrangement is optimized for allowing a small lateral distance betweenadjacent optical channels in order to economically exploit the diereal-estate area, consequently the lens diameters and the wall-thicknessof (opaque) spacer structures may be reduced. However, for the fieldflattening structure itself this is sometimes difficult to achieve. Thereason for this is that the proximity of this lens surface to the imagesensor requires a lens having a diameter on the order of magnitude ofthe image circle (scaled by the distance between the two). In order torelax this requirement the field flattener can be designed andimplemented in a non-rotational-symmetric way. This results in arectangular rather than circular footprint of this lens surface. Thusthe lens would have a large lateral extension along the corners (=imagesensor diagonal), thereby allowing multiple lens surfaces within onearray to be situated much closer together in x-y and thus allowing anoverall smaller pitch between the channels. Third, optical channelswithin one array dedicated to imaging different “colors” (parts of theoverall wavelength spectrum to be captured) may differ in the particularsurface profile of at least one lens surface. The differences in thesurface profiles of those lenses in one array can be minor, but are veryeffective in order to keep the back focal length (“BFL”)color-independent, and consequently allow (almost) equally sharp imagesfor the different colors without the costly need for wide-spectral-bandachromatization. Moreover, after computational color-fusion ahigh-resolution polychromatic image can still be achieved. Here,preferably the first surface of the first lens element would bespecifically optimized for the narrow spectral band of the respectivecolor channel.

The benefits of this array-dedicated design of the single channels ofthe array camera include:

-   -   Extremely high image quality, both in terms of resolution and        contrast, and image quality homogeneity over the field of view        (close to diffraction limited performance and close to        isoplanatically);    -   A reduced TTL;    -   A reduced fabrication tolerance sensitivity due to reduced ray        bending;    -   Low color cross-talk, due to color filters now being        differentiating the different optical channels rather than        having a Bayer pattern on the pixel level;    -   Low pixel cross-talk, due to smaller pixel stack height); and    -   Reduced color inhomogeneity, due to the color filters being far        from the image plane.

Finally, of particular note in this design is the fact that the separatecolor channels only need to be optimized for their respective spectralbands. This results in overall higher polychromatic resolution, whileminimizing the need for achromatization correction within the individualchannels. This in turn leads to the ability to implement simpler overallaberration balancing or correction process, and therefore have simplerlenses and lens manufacturing processes, and/or better MTF, and/or lowerF/#. In many embodiments the MTF characteristics of the five-surfaceoptical arrangement allow for contrast at spatial frequencies that areat east as great as the resolution of the high resolution imagessynthesized by the array camera, and significantly greater than theNyquist frequency of the pixel pitch of the pixels on the focal plane,which in some embodiments may be 1.5, 2 or 3 times the Nyquistfrequency.

Embodiment 3 Monolithic Lens Design with Embedded Substrate

The embodiments previously discussed dealt with lenses made inaccordance with a polymer on glass WLO process. In the followingembodiment optical arrangements and designs using a monolithic lens WLOprocess are provided. In particular, in a first embodiment a monolithiclens stacked with planar substrates for use in forming apertures andfilters is described.

FIG. 6A shows the current state of the art of monolithic lens systems.More or less the same conceptual approach is taken as in creatinginjection-molded lenses and their packaging. In the state of the art ofmonolithic lens WLO, many lenses are fabricated on a wafer scale. Thesereplicated lenses 600 are stacked with other previously replicated lenswafers of different topology, the sandwich is diced, and the lens cubesare packaged into an opaque housing 602 with the image sensor 604, whichcontains the aperture stop 606 at the front as shown in FIG. 6A. Thisvery much limits the degrees of freedom available for the optical designof the objective. In addition, it makes it difficult to accuratelyreplicate and align the lenses with respect to each other, particularlyas it is difficult to determine precisely the placement of the aperturestop. Moreover, from the standpoint of optical design it is verydesirable to have the aperture stop between the two lens elements, notin front of the first lens element as shown in FIG. 6A. Currently, asshown in FIG. 6B, the only method for forming apertures of this type onmonolithic lenses is to use a highly imprecise screen-printing method inwhich apertures 608 in opaque resins are printed onto the flat portionsof the lens interfaces. The lateral accuracy of those apertures isunsuitable for their use as a system stop, which must be preciselyaligned with the lenses.

In short, although monolithic lens WLO is potentially an attractivemeans to manufacture cheap miniaturized optics for array cameras, thecurrent monolithic systems are directly adapted from the methods used toform lenses by injection molding. As a result, many of the techniquesused in conventional polymer-on-glass WLO to ensure proper alignment arenot applied, leading to alignment accuracy problems as well as to alimited lens design space. The current embodiment is directed to a novelmethod of forming monolithic lenses that combines the monolithic WLOlenses with substrates that hold apertures and additional structures inprecise alignment, thereby reducing the limitations of conventionalmonolithic lens WLO.

An exemplary embodiment of the method of monolithic lenses formed inaccordance with the invention is shown in FIG. 6C. As shown, in thisembodiment, monolithic lenses 612 & 614, fabricated by an independentreplication process, are stacked with a substrate or sheet 616 thatholds apertures 618 & 620. (As discussed previously, it will beunderstood that the monolithic lenses may be formed of glass orpolymer.) Because the apertures can be formed on the substrate withlithographic precision, it is possible to align the elements withsufficient lateral precision to function as the aperture stop. Inaddition, although not shown in FIG. 6C, the accuracy of the alignmentin such a system is increased by cooperative alignment marks, which aredisposed in the opaque layer(s) where the transparent openings for theapertures are structured, to provide a guide for the precision alignmentof the lenses and apertures. In particular, in a wafer stack formed froma series of wafer surfaces, themselves formed from the elements of anumber of optical arrangements, alignment marks would be formed inrelation to each wafer surface. Each of the alignment marks would becooperative with an alignment mark on an adjacent wafer surface suchthat when cooperatively aligned the alignment marks would aide in thelateral and rotational alignment of the lens surfaces with thecorresponding apertures. Using these alignment marks results in a veryhigh lateral alignment accuracy (on the order of a few μm) compared tohaving the aperture stop in the external housing, which results in anaccuracy of several 10-20 μm.

In addition to apertures, the current method of providing a substrateembedded into monolithic lenses provides a base onto which any number ofdifferent structures, coatings, kinds of substrates or sheets can beapplied in order to achieve a desired optical functionality. A number ofthese possibilities are shown in FIG. 6D, these include where there aretwo apertures on the front and back of the substrate that are the samesize (6D1) or different sizes (6D2); where an additional IRCF coating,such as a homogenous IR cut-off filter made by a dielectric interferencecoating, is applied on either one or both sides of the substrate (6D3);where an additional color filter array material coating is applied tothe substrate (6D4); where the sheet or substrate contains an adaptiverefractive optical element allowing for the adjustment of the opticalpower of the element by changing an applied voltage, which can allow forthe focusing of the whole lens stack, accounting for fabricationtolerances (such as BFL variations)(6D5); or where the sheet orsubstrate is made from an opaque material (6D6).

Other alternative designs that may be incorporated into the substratesand monolithic lenses of the instant invention, but that are not shownin the figures may include:

-   -   A substrate that is made of a material that is itself an        absorptive IRCF (or combined with a dielectric coating);    -   A structured dielectric IRCF complemented by a structured        dielectric NIR-pass filter for extended color camera modules;    -   A polarization filter disposed on the surface of the substrate        or that is preformed into the sheet;    -   A thin diffractive lens applied to the surface of the thin        substrate by replication of an additional thin polymer layer, or        also by etching the diffractive structure into the glass, front        and/or backside of the substrate surface (See Embodiment 9);        and/or    -   Standoffs or spacing structures integrated into the monolithic        lenses in addition to the actual lens surfaces in order to        provide the correct positioning between the lens surfaces and        the system aperture on the thin substrate (See Embodiment 8).

Although the above has focused on specific substrate structures andadditional optical elements that the substrate embedded monolithiclenses of the instant inventions can incorporate, it should beunderstood that there are other features unique to the embeddedsubstrates of the invention. For example, unlike conventional polymer onglass WLO where substrates or sheets must be sufficiently thick to allowreplication of lenses thereon, the embedded substrates or sheets of theinstant invention can be thin in comparison to wafer level opticsstandards since there is no need to replicate lenses on them. As aresult, the mechanical stability and stress applied to the substrate isnot an issue. In contrast, the independently replicated monolithiclenses can themselves serve to stabilize the glass substrate. Moreover,this holds true even for a singlet lens construct (i.e., one monolithiclens and one thin substrate).

Finally, while individual modifications to the basic embedded substratemonolithic lens optical array are described above, it should beunderstood that all or some of these features may be applied in variouscombinations to the substrates to obtain the desired functionality ofthe optical arrangement. In particular, these structures may allow forthe implementation of optical arrangements that allow for contrast atspatial frequencies that are at least as great as the resolution of thehigh resolution images synthesized by the array camera, andsignificantly greater than the Nyquist frequency of the pixel pitch ofthe pixels on the focal plane, which in some embodiments may be 1.5, 2or 3 times the Nyquist frequency.

Embodiment 4 Monolithic Lens Design with Embedded Aperture Stop

This embodiment of the invention provides yet another alternative foraperture and filter placement within the lens stack of polymer or glassWLO monolithic lenses. As described above with respect to Embodiment 3,the current state of the art for producing monolithic lens opticalarrays is to stack the independently replicated monolithic lens wafers,dice the sandwich and package the lens cubes into an opaque housingwhich contains the aperture stop as an integral part at the front of thearray. This methodology limits the degrees of freedom for the opticaldesign of the objective, as well as making it extremely difficult toaccurately align the lenses with respect to of the aperture stop.

Embodiment 3 of the invention described a polymer or glass monolithiclens stacked with substrates for the placement of apertures and filters.In that embodiment of the invention, a substrate, such as glass, havingaperture and/or filters thereon is disposed between separatelyfabricated monolithic lenses. This novel optical arrangement providesaddition degrees of freedom for the optical design, and increases thelateral precision of the lens-aperture-alignment. The inventiondescribed in this embodiment embeds apertures and filters directlywithin a monolithic lens (See FIGS. 7A to 7G), providing even more anddifferent degrees of freedom for the optical design, while maintaining ahigh lithographic precision for the lateral aperture placement.

As discussed with respect to Embodiments 1 to 3, lithographic proceduresfor producing apertures and/or filters are well known for polymer onglass WLO, e.g., spin on photoresist, expose desired areas through acorrespondingly structured photomask, develop unexposed orexposed—depending on whether a positive or negative photoresist isused—areas away; either the photoresist itself is the opaque layer theapertures are structured in, or the (CFA) filter; or the photoresist isa protective layer for a previously applied metal or dielectric coating,which prevents the etching away of that material at the desired areaswhen the wafer is placed into an etchant. However, for a monolithic lenstypically the monolithic lenses are replicated as double-sided lenses.As a result of the unusual topography, these WLO techniques cannot beapplied since a plano surface is needed for lithography.

The current invention is directed to an optical arrangement and processfor producing such monolithic lenses formed of either polymer or glasswith embedded apertures and filters. One embodiment of the invention isshown schematically in FIG. 7A. As shown, in this embodiment, first thethick front-side of the lens 702 is replicated as a plano-convex orplano—concave element. Preferably, the front-side stamp, which alsoholds the lens profiles, additionally contains alignment marks (asdescribed above in Embodiment 3) that are further used in the othermanufacturing steps to aide in the precise alignment of the variouselements to the overall optical arrangement. Because the backside 704stamp in this initial step may be simply a highly flat and/or highlypolished plate, no precise lateral alignment of the two stamps isrequired, only wedge error compensation as well as the correct thicknessneeds to be ensured. These modest requirements simplify this initialprocess step considerably.

Once the first lens element of the arrangement is complete, apertures706 and filters 707 are applied on the plano back-side 704 of this lenselement. As these apertures must be precisely aligned, it is preferableif the front-side of the lens element is provided with alignmentfeatures (not shown) that can be used during manufacture to assist inpositioning the apertures with respect to the lens by aligning thealignment marks in the photomask of the apertures and/or filters to thecomplementary alignment marks within the first lens layer. Alignmentmarks, which may be of any suitable design, provide the benefit ofallowing much higher lateral alignment accuracy (few μm) compared tohaving the aperture stop in the external housing, which has a typicallateral alignment precision of several 10-20 μm.

Once the apertures/filters 706/707 are positioned on the back-side 704of the first lens surface, the second lens surface 708 is replicated onthe plano back-side 704, of the first lens surface 700. This second lenssurface can be aligned either based on the alignment features in thefirst lens front-side 702, or based on alignment features within theaperture layer. However, it should be understood that aligning thesecond lens surface to the front-side of the first lens surface ispreferred since the precision is expected to be better due to reducederror propagation when referring to this initial surface.

Although FIG. 7A provides an embodiment of a desired lens element withan embedded aperture in accordance with the current invention, a numberof modifications or additional elements may be incorporated into theinvention. For example, multiple filters 712, even having differentphysical natures, can be stacked on each other as shown in FIG. 7B. Itshould be understood that any desired filter may be applied in thismanner, including, for example, a CFA filter or a structured IRCFfilter.

In addition, although the materials used in forming the first 714 andsecond lens 716 surfaces have not been specified, it should beunderstood that the different replications may be formed from anysuitable material, and that the material may be the same for bothreplications (as shown in FIG. 7C) or two different materials (as shownin FIG. 7D). If the same lens materials are used for the first andsecond replication, the inside lens surface optically vanishes (in otherwords: it is not visible to the light and thus provides no refractionand consequently is free of any Fresnel reflection losses). However,making the two replications from two different materials provides yetanother degree of freedom in manufacturing the optical arrangements,especially for achromatization correction if the Abbe numbers of the twomaterials are different (See Embodiment 6).

Although only a single lens element of a single lens channel of apotential array camera or wafer arrangement is shown, it should beunderstood that the monolithic lenses (polymer or glass) formed inaccordance with the current invention may be duplicated as necessary toform the plurality of lens stacks needed for the array camera, and thatthe monolithic lenses may be combined with other lens elements torealize an optical arrangement having the desired characteristics. Forexample, FIGS. 7E, F and G provide schematic diagrams for monolithiclens arrangements suitable for array camera architectures. FIGS. 7E and7F show two different monolithic doublet designs, while FIG. 7G shows atriplet design. The arrows in the diagrams indicate where an embeddedsystem aperture or “stop” has been disposed between the monolithiclenses on one of the planar surfaces of the monolithic lenses.

In summary, while there is no doubt that monolithic lens WLO is veryattractive for manufacturing optics for cheap miniaturized cameras,current methods are adapted directly from techniques used for injectionmolded lenses. As a result, several benefits of polymer-on-glass WLO arenot used, leading to alignment accuracy problems as well as to limitedlens design degrees of freedom. The combination of monolithic lenses andlithographic technologies described in the current embodiment allows forthe manufacture of precise apertures and additional structures formonolithic lenses and their alignment to the monolithic lenses. This, inturn, allows for greater flexibility in the choice of the z-position foraperture stop and filters, increased lateral accuracy of thelens-aperture alignment when compared to conventional stops that areintegrated into the lens housing, and the plano intermediate surface ofthe monolithic lens allows application of lithographic technologies forstructuring the apertures while maintaining the benefits of themonolithic lens over the polymer on glass WLO. In particular, thesestructures may allow for the implementation of optical arrangements thatallow for contrast at spatial frequencies that are at least as great asthe resolution of the high resolution images synthesized by the arraycamera, and significantly greater than the Nyquist frequency of thepixel pitch of the pixels on the focal plane, which in some embodimentsmay be 1.5, 2 or 3 times the Nyquist frequency.

Embodiment 5 Three-Element Monolithic Lens Design

This embodiment of the invention provides yet another alternative foraperture and filter placement within the lens stack of polymer or glassWLO monolithic lenses. As described above with respect to Embodiments 3and 4, the current state of the art for producing monolithic lensoptical arrays is to stack the independently replicated monolithic lenswafers, dice the sandwich, and package the lens cubes into an opaquehousing which contains the aperture stop as an integral part at thefront of the array. This methodology limits the degrees of freedom forthe optical design of the objective, as well as making it extremelydifficult to accurately align the lenses with respect to the aperturestop.

As described in both Embodiment 3 and 4, a major problem of themonolithic lens process is that there is no suitable method to provide aprecise system aperture (array) as well as (color- or IR cut-off-)filters within the lens stack of WLO monolithic lenses. The currentembodiment provides another alternative to bring a lithographicallyfabricated aperture (stop), as well as filters into a polymer or glassWLO monolithic lens stack. In particular, this embodiment builds on thedesign introduced in Embodiment 4, in which one element of the lensdesign is forced to have a plano surface where the aperture and filterscan be lithographically structured. As described above, in such anembodiment the plane side of the either plano-convex or plano-concaveelement can be used as substrate for the subsequent lithography step.The current embodiment provides a three-element optical arrangementusing this plano-element monolithic design.

As shown schematically in FIG. 8A, the basic three-element design of theinstant embodiment is characterized by the following properties:

-   -   A first plano-convex lens element 800 that has a convex first        surface 802 as well as a plane second side 804 carrying the        system aperture stop as well as required filter structures.        Preferably, this first element is made from a first (low        dispersion, low refractive index) lens material.    -   A second concave-convex lens element 806 that has a concave        first surface 808, bent towards the object side and a convex        second surface 810, where the concave first surface 808 is very        shallow and this concave surface is very close to the plane        (second) surface 804 of the first element 800. Again, preferably        this lens element is made from a first (low dispersion, low        refractive index) lens material. In addition, in a preferred        embodiment, the surface profile of this shallow concave surface        808 close to the system aperture stop 804 is the one        optimized/adapted to the specific narrow spectral band of the        different color channels of an array camera.    -   A third menisc-lens element 812 that has a concave first surface        813 and a convex second surface 814, both bent towards the        object side. This lens is preferably a strongly bent        concave-convex lens that is made from a second (high dispersion,        high refractive index) lens material. This third lens element is        disposed adjacent to the image sensor cover glass 816, which        itself is placed in above the image sensor 817.

This design has two significant advantages, first, a plane,substrate-interface-like, surface (e.g., surface 804 in FIG. 8A) isintroduced in the lens stack. This plane,substrate-interface-like-surface can be used to apply a highly accurate(sub- or few-micron centering tolerance) aperture stop byphotolithography. This is a major improvement in precision as thecurrent state-of-the-art (screen printing)) has a centering tolerance ofaround 20 μm, which is insufficient for high image quality arraycameras. In addition, color filters (CFA) and/or dielectric filters(IRCF) or other structures, which need a planar substrate, can beapplied to this planer surface.

Second, the design provides a surface which is very close to theaperture stop (first surface of the second element) whose surfaceprofile can be optimally adapted to the specific narrow spectral band ofthe different color channels of an array camera, as will be described ingreater detail below with reference to the data-plots in FIGS. 8E to 8J,below.

Although one specific embodiment of the three-element monolithic lensdesign is shown in FIG. 8A, it should be understood that there are manydifferent implementations of the above general design principle, asshown and described in FIGS. 8B to 8D, below. In particular, FIG. 8Bshows a modification of the basic optical arrangement in which thecurvature at edges 818 of the second surface 820 of the second lenselement 806, and at the edges 820 of both surfaces of the third lenselement 812 quickly change slope towards these edges. Such a design hasthe benefit of allowing for a decrease in the steepness at the edge ofthe first surface of the third element. FIG. 8C shows a modification ofthe basic optical arrangement in which the first element 800 is thinnerand the third element 812 is in consequence made thicker. Such a design,however, requires an increase in the steepness of the first surface 822of the third element 812. FIG. 8D shows a modification of the basicoptical arrangement in which the first element 800 is thinner and thethird element 812 is thicker, and where the curvature of the surface atthe edge 818 of the second surface 820 of the second element 806 quicklychanges slope towards the edge formed thereof. Again, this design hasthe benefit of decreasing the steepness of the first surface of thethird element 812.

In another alternative embodiment that can be applied to any of thearrangements described above, the first lens element can be made as apolymer-on-glass wafer level lens instead of a (polymer or glass)monolithic lens. This would mean that there would be a (comparativelythick) glass substrate where the aperture stop and filters would belithographically applied to the second side thereof, and the first lenssurface would be replicated on the first side. This “hybrid lens” wouldthen be stacked with the second and third lens elements, which wouldboth be fabricated by a monolithic lens process. Alternatively, thesecond lens element could be a hybrid lens in which the polymer lenssurfaces would be replicated on both sides of a thinner glass substrate.However, the third lens element would always be monolithic due to themenisc-nature of this lens. There are several advantages of thiscombination of technologies, namely:

-   -   The first lens is a comparatively thick element with a plane        backside and a shallow front lens surface, so little is lost        functionally by inserting the glass substrate.    -   The use of the substrate provides additional        robustness/stability/planarity during the application of the        aperture and filters due to the presence of the glass substrate.        In addition, the first lens surface quality can be improved due        to the stable glass substrate it is replicated on.    -   There is less (especially lateral) thermal expansion than with a        purely monolithic lens since the thick glass substrate with        about 1/10^(th) of the CTE of the polymer serves as a permanent        carrier of the overall lens stack providing the majority of the        mechanical integrity.

FIGS. 8E to 8J provide data plots showing the optical properties ofthese novel three-element monolithic optical arrangements. Inparticular, FIGS. 8E to 8H, provide plots of MTF vs field (8E), Strehlratio vs. field (8F), distortion and field curvature (8G) as well as MTFvs. spatial frequency (8H) of the lens design shown in FIG. 8A for agreen channel. Meanwhile, FIGS. 8I & 8J provide plots of MTF vs field(8I) and Strehl ratio vs. field plots (8J) of the corresponding bluechannel of the design shown in FIG. 8A. It should be noted that only thesurface profile of the first surface 808 of the second element 806 needsto be altered to optimize the optical arrangement for a different colorchannel. As can be seen from this data, the three-element monolithicoptical arrangement provides high image quality (See, e.g., FIGS. 8E to8H) comparable to that of a design using a field-flattening element(such as e.g. applied in Embodiment 2 above). Moreover, because onlythree lens elements need to be stacked in the current design it is muchmore suitable for manufacture using a monolithic method compared tocomplex conventional multi-element optical arrangements.

The lens material sequence (i.e., in the above embodiment high Abbenumber, high Abbe number, low Abbe number) for the positive, positive,negative elements provides an efficient way of achromatization for eachconsidered channel's spectral band (See Embodiment 6). For example, evenfor regular dispersion materials the blue channel performance seen inthe exemplary embodiment is much better than can be obtained for regulardesigns (See, e.g., FIGS. 8I and 8J). Moreover, even though for arraycameras each channel only has to perform well for a comparatively narrowspectral band, this achromatization still increases the performancesince both the central wavelength and the wavelengths at the sides ofthe used spectral band of the considered channels are imaged sharply.

In many embodiments the MTF characteristics of the three elementmonolithic optical arrangement allow for contrast at spatial frequenciesthat are at least as great as the resolution of the high resolutionimages synthesized by the array camera, and significantly greater thanthe Nyquist frequency of the pixel pitch of the pixels on the focalplane, which in some embodiments may be 1.5, 2 or 3 times the Nyquistfrequency.

Embodiment 6 Different Lens Material Sequences for Channels that Workwith Different Spectral Bands

Although the above embodiments have focused on specific opticalarrangements, it will be understood that the current invention is alsodirected to novel methods and materials for modifying the opticalproperties of the various lens elements of these novel opticalarrangements. For example, in a first such embodiment, the invention isdirected to the use of different lens materials (or combinationsthereof) for different color channels.

As shown in FIGS. 9A, 9B and 9D, using conventional array optics,channel specific color-focusing in an array camera so far is limited toadjusting at least one surface, (i.e., the surface profiles of front-and/or backside for lens 900 or lens 902) for channel-specificcorrection of the back-focal-length (BFL) for axial color. However, thematerial sequence for the different elements of the lens channels isalways the same, independently of the color the considered channel issupposed to work for. It should be noted that the only differencebetween the lens arrays in FIGS. 9A and 9B is that in FIG. 9B thesupporting structure of the array is opaque and going through the fulllength of a channel. (For reference the specific color channel red “R”,green “G”, or blue “B” is indicated by the letter in the schematicsprovided.) However, while there are several materials with highrefractive index, which are beneficial in achieving strong refractivepower with shallow lens profiles, these materials usually also show highspectral dispersion. In particular, typically one has a choice betweenhigh refractive index and low Abbe number (high dispersion) materials(“flint-like”), and low refractive index and high Abbe number (lowdispersion) materials (“crown-like”). Indeed, for (lens-) polymers theabove connection is always valid, dispersion always increases withincreasing refractive index. If this physical connection of the twomaterial properties was not the case, from an optical design standpointthe choice would usually be made to use a high index material (so thatthe surface of the lens can be shallow, while still maintaining strongoptical power) with low dispersion (so that the difference in refractivepower for different wavelengths would be small). However, as statedabove, such polymer materials are not available, so one has to choose ifthe priority is on either one of the two properties.

While using a flint-like material can be acceptable for the green andred channels, it can impact the blue spectral band disproportionately,because dispersion is related to the change of refractive index withwavelength and usually this change is stronger in the blue spectral bandthan in the green and red ones. In short, while green and red channelswould profit from the use of such a high index material, the bluechannel would show too strong axial color aberration due to the relatedlarge dispersion. The current embodiment takes advantage of the arraynature of the camera to allow the use of a different material sequencein the blue channel (as shown in FIG. 9C), which may be less optimalwith regard to refractive index, but shows much less spectraldispersion. Using such a method makes it possible to adapt one or morelens profiles to optimize a channel to its respective spectral band, andto optimize the material sequence used, e.g., here changing the materialsequence for the blue channel.

It should be understood that the ability to modify the material sequenceto optimize it for a specific color channel may be used in injectionmolded lenses (as shown in FIG. 9C) or with a specific type of polymeron glass “WLO” lenses (as shown in FIGS. 9D and 9E) (where the lensmaterial is dispensed in separated islands prior to the replication(e.g. by some device similar to an ink jet) other than with the waferscale puddle dispense). For example, in an injection molding process a“crown-like” polymer material would, e.g., be PMMA, Zeonex (COP) andTopas (COC), and a “flint-like” material would be Polycarbonate (PC) andPolystyrene (PS). Finally, as described in reference to the inventionmore broadly, the material sequence may also be modified in glass moldedlenses as well.

Embodiment 7 Polymer on Glass WLO Novel Aperture Stop

Again, although the above embodiments have focused on specific opticalarrangements, it will be understood that the current invention is alsodirected to novel methods and materials for modifying the opticalproperties of the various lens elements of these novel opticalarrangements. In a second such embodiment, the invention is directed toa novel arrangement that could be used in any polymer on glass WLO, inwhich the aperture stop is disposed on a separate substrate in the airspacing between lenses.

As shown schematically in FIG. 10A, in the conventional polymer on glassWLO, apertures 1000, and in particular the aperture stop, is structuredon the supporting glass substrate 1002, and then the lenses 1004 and1006 are replicated above the aperture. In the current embodiment, anadditional layer 1010 is introduced between the lens substrates 1012 and1014, upon which the aperture stop 1016 is disposed. In such anembodiment, the apertures may be made using any suitable technique, suchas, for example, transparent openings in an opaque layer (e.g. metal,metal oxide or opaque photoresist) on a thin (glass) substrate, be(metal) etch mask, etc. Positioning the aperture in the air spacebetween the lenses as an additional diaphragm, or as an aperture on verythin (glass) sheet, rather than forcing it to be on the substrate underthe polymer lens yields a number lens designs benefits in terms of MTFperformance. In contrast, constraining the apertures to the substratesurfaces for a large variety of lens designs reduces performance by5-10% over the full field.

Embodiment 8 Polymer Injection- or Precision Glass Molded Lens Arrays

Again, although the above embodiments have focused on specific opticalarrangements, it will be understood that the current invention is alsodirected to novel methods of manufacturing the various lens elements ofthese novel optical arrangements. In a third such embodiment, theinvention is directed to a novel method of manufacturing opticalarrangements for use in camera arrays in which stand-offs and mechanicalself-alignment features for assembly are included in the manufacture ofthe lenses.

In conventional polymer injection- or precision glass moldingtechniques, a cavity for producing one lens array (front and back side)is provided. The mold cavity is filled with a suitable material, suchas, for example, PMMA or polycarbonate for polymer injection molding orpreferably “low-Tg-glasses” such as e.g. P-BK7 or P-SF8 for precisionglass molding. Then for conventional camera assembly alignment barrelsare used in which the molded lenses are stacked and glued together. Inan array camera this method does not provide sufficient alignmentprecision. The current invention proposes a method in which mechanicalalignment features are provided in the lens mold. In other words, duringthe polymer injection- or precision glass molding process, not only thelens features are replicated into the array, nor even optical alignmentmarks, but also small mechanical features are formed into the front- andback-faces of the elements, which allow mechanical self-alignment withthe adjacent array, such as, for example, complementary rings andspherical segments, pins and holes, cones and pyramids withcomplementary (and correspondingly shaped) cavities on the opposingelement.

Two such embodiments are shown in FIGS. 11A and 11B. For example, FIG.11A shows an example in which spacing structures/stand-offs 1100 areincluded in injection molding process of the lens array 1102. Forpolymer injection molding it is desirable in such an embodiment that thematerial combination choice and spacer thickness provideathermalization. In short, it is desirable that the do/dT of the lensmaterial is compensated by the CTE of spacer. Alternatively, the sametechnique may be used in independently fabricated spacer or hole matrixstructures. In such an embodiment, as shown in FIG. 11B an (opaque)cavity array 1104 is used as the supporting structure into which thelenses 1106 are replicated.

Embodiment 9 Waveplate or Multilevel Diffractive Phase Elements

In yet another embodiment, the invention is directed to a waveplate ormultilevel diffractive phase element (“kinoform”) for channelwisecorrection of chromatic aberrations in an array camera, and an iterativefabrication process tolerance compensation.

Currently one of the three or four lens surfaces in the objective ischannel-wisely optimized in order to correct for chromatic aberrationsof the specific channel (see, e.g., U.S. Pat. Pub. No.US-2011-0069189-A1, the disclosure of which is incorporated herein byreference). For this a special mastering regime for the array tool isrequired since slightly different lenses need to be fabricated withinone array. The overall lens property can be considered as the sum of theaverage required shape, and the individual color correction. However,the total profile has to be implemented by machining, which addsdifficulty for diamond turning mastering techniques. (See, e.g., U.S.patent application Ser. No. 13/050,429, the disclosure of which isincorporated herein by reference.)

Lens design experiments show that it could be beneficial to separate thechannel-averaged optical power from the channel-specific optical power,which is then related to the color correction. The current embodiment isdirected to an optical arrangement that accomplishes this channel-wisecorrection using a channel-specific surface that introduces only a minorwavefront deformation of exactly the size needed to distinguish thechannels from each other so that they are perfectly adapted to theirindividual waveband. The wavefront deformation required for this istypically on the order of only several wavelengths. As a result, thissurface can either be a very shallow refractive surface (“waveplate”), a“low frequency” diffractive lens (“kinoform”) or radial symmetricmultilevel diffractive phase element. As a result, there is no longer aneed to machine slightly different lenses (e.g. by diamond turning), butall the lens surfaces within one array could be equal. In addition,different technologies can be used for the origination of arrays of suchchannel specific surfaces, including, “classical” lithographicmicrooptics fabrication technologies such as laser beam writing, grayscale lithography, E-beam lithography, binary lithography, etc.Moreover, these techniques are more suitable for manufacturing slightdifferences in the surfaces comprised in the array, they have muchhigher lateral precision than mechanical origination means, and theyprovide much higher thickness precision (i.e., phase accuracy of thesurface).

In addition, it is possible to use the above advantages to compensatefor the effects of systematic fabrication errors on image quality. Aflow chart of this manufacturing method is provided in FIG. 12. Asshown, in a first step the optical channels of the array camera aredesigned. This at this time includes the nominal shape of the waveplateor multilevel diffractive phase element which is used for channel-wisecolor aberration correction only. In a second step the array lens moduleis fabricated by a suitable means (as described above). Then in stepthree, the systematic deviation of the lens prescriptions from designexpectations are determined by lens surface metrology, centering- anddistances-measurements are performed, as well as the systematicdeviation of optical performance from design expectation areexperimentally determined. The module is then redesigned (Step Four) byadapting the above aberration correcting surfaces in order to compensatefor all determined systematic errors elsewhere in the stack (profile,xy-position, thickness, etc.). In step six the array lens module isre-fabricated. And finally, the back focal length is used as a lastcompensator for all remaining systematic deviations (Step Seven). Theadvantage of this method is that there are more degrees of freedom,rather then being able to change back focal length only, as is the casein conventional system. This leads to better overall performance,potentially without impacting optical magnification.

General Considerations

Finally, it will be understood that in any of the above embodiments,multiple identical or slightly varied versions of such opticalarrangements may be collocated next to each other in an array. Thevariation of the optical arrangements within an optical array is relatedto e.g. one of the following optical performance parameters of theconsidered channel: “color” (identifying the narrow spectral band theconsidered optical channel is supposed to image of the overall spectralband the whole system shall image), e.g. RGB (and NIR), field of view(FOV), F/#, resolution, object distance, etc. Most typical is thedifferentiation into different colors, but different FOVs, for example,would allow for different magnifications while different F/#s wouldallow for different sensitivities and so forth. In many embodiments theMTF characteristics of the optical arrangements are configured to allowfor contrast at spatial frequencies that are at least as great as theresolution of the high resolution images synthesized by the arraycamera, and significantly greater than the Nyquist frequency of thepixel pitch of the pixels on the focal plane, which in some embodimentsmay be 1.5, 2 or 3 times the Nyquist frequency.

DOCTRINE OF EQUIVALENTS

While particular embodiments and applications of the present inventionhave been illustrated and described herein, it is to be understood thatthe invention is not limited to the precise construction and componentsdisclosed herein and that various modifications, changes, and variationsmay be made in the arrangement, operation, and details of the methodsand apparatuses of the present invention without departing from thespirit and scope of the invention as it is defined in the appendedclaims.

What is claimed is:
 1. An array camera, comprising: a plurality ofcameras, where each camera includes separate optics, and a plurality oflight sensing elements; a processor; wherein the optics of each of theplurality of cameras are configured so that each camera has a field ofview that is shifted with respect to the field-of-views of the othercameras and so that each shift includes a sub-pixel shifted view of thescene; wherein the light sensing elements of a given camera in theplurality of cameras have a pixel pitch defining a camera Nyquistfrequency, and where the optics of the given camera have a modulartransfer function (MTF) such that the optics optically resolve contrastat spatial frequencies higher than the camera Nyquist frequency (Ny);and wherein software directs the processor to: obtain a set of lowresolution images from the plurality of cameras, where each of the lowresolution images include aliasing patterns; determine disparity betweenpixels in the set of low resolution images to generate a depth map froma reference viewpoint, where the depth map indicates distances tosurfaces of scene objects from the reference viewpoint; synthesize ahigh resolution image using the set of images and the depth map, wherethe spatial frequency at which the high resolution image displayscontrast is greater than the camera Nyquist frequencies (Ny) of theplurality of cameras and less than the spatial frequencies at which theoptics of each camera optically resolve contrast.
 2. The array camera ofclaim 1, wherein the aliasing patterns in each image in the set of thelow resolution images include differences due to the different sub-pixelshifted views of the scene provided by the optics of the plurality ofcameras.
 3. The array camera of claim 1, wherein the software furtherdirects the processor to synthesize a high resolution image by:determining scene dependent geometric corrections to apply to the pixelsfrom each of the images within the set of low resolution images toeliminate disparity; and fusing the set of low resolution images usingthe scene dependent geometric corrections.
 4. The array camera of claim3, wherein the software further directs the processor to perform superresolution processing to reconstruct the high resolution image using thefused image, the scene dependent geometric corrections, and the set oflow resolution images.
 5. The array camera of claim 1, wherein the MTFof the optics of a given camera in the plurality of cameras is such thatthe optics optically resolve contrast at spatial frequencies at least1.5 times the camera Nyquist frequency Ny.
 6. The array camera of claim1, wherein the MTF of the optics a given camera in the plurality ofcameras is such that the optics optically resolve contrast at spatialfrequencies at least 2 times the camera Nyquist frequency Ny.
 7. Thearray camera of claim 1, wherein the MTF of the optics a given camera inthe plurality of cameras is such that the optics optically resolvecontrast at spatial frequencies at least 3 times the camera Nyquistfrequency Ny.
 8. The array camera of claim 1, wherein the MTF of theoptics a given camera in the plurality of cameras is such that theoptics optically resolve contrast at spatial frequencies at least 10%greater than the camera Nyquist frequency Ny multiplied by the ratio ofthe resolution of the high resolution image to the resolution of theimages in the set of low resolution images.
 9. The array camera of claim1, wherein the MTF of the optics a given camera in the plurality ofcameras is such that the optics optically resolve contrast at spatialfrequencies at least 20% greater than the camera Nyquist frequency Nymultiplied by the ratio of the resolution of the high resolution imageto the resolution of the images in the set of low resolution images. 10.The array camera of claim 1, wherein the MTF of the optics a givencamera in the plurality of cameras is such that the optics opticallyresolve contrast at spatial frequencies at least 30% greater than thecamera Nyquist frequency Ny multiplied by the ratio of the resolution ofthe high resolution image to the resolution of the images in the set oflow resolution images.
 11. The array camera of claim 1, wherein theoptics of each camera in the plurality of cameras comprise athree-surface optical arrangement comprising: a first lens elementhaving a first convex proximal surface and a first concave distalsurface, wherein the diameter of the first convex surface is larger thanthe diameter of the first concave surface; and a second lens elementhaving a substantially flat second proximal surface and a second convexdistal surface, wherein the diameter of the flat second proximal surfaceis smaller than the diameter of the second convex surface, and whereinthe diameter of the second convex surface is intermediate between thediameters of the first convex surface and the first concave surface;wherein the first and second lens elements are arranged sequentially inoptical alignment with an imager positioned at the distal end thereof.12. The array camera of claim 1, wherein the optics of each camera inthe plurality of cameras comprise a five-surface optical arrangementcomprising: a first lens element having a first convex proximal surfaceand a first concave distal surface, wherein the diameter of the firstconvex surface is larger than the diameter of the first concave surface;a second lens element having a second concave proximal surface and asecond convex distal surface, wherein the diameter of the second concaveproximal surface is smaller than the diameter of the second convexsurface; and a third lens element having a third concave proximalsurface and a third planar distal surface, wherein the diameter of thethird concave proximal surface is larger than the diameters of any ofthe surfaces of the first and second lens elements; wherein the first,second and thirds lens elements are arranged sequentially in opticalalignment with an imager positioned at the distal end thereof.
 13. Thearray camera of claim 1, wherein the optics of each camera in theplurality of cameras comprise a substrate embedded hybrid lens opticalarrangement comprising: a substrate having proximal and distal sides; afirst monolithic lens element having first proximal and distal surfacesdisposed on the proximal side of said substrate; a second monolithiclens element having second proximal and distal surfaces disposed on thedistal side of said substrate; and at least one aperture disposed onsaid substrate in optical alignment with said first and second lenselements; wherein the first and second lens elements are arrangedsequentially in optical alignment with an imager positioned at thedistal end thereof.
 14. The array camera of claim 1, wherein the opticsof each camera in the plurality of cameras comprise a monolithic lensoptical arrangement comprising: at least one lens element comprising: afirst monolithic lens having first proximal and distal surfaces, whereinthe first proximal surface of the first monolithic lens has one ofeither a concave or convex profile, and wherein the first distal surfaceof the first monolithic lens has a plano profile; at least one aperturedisposed on the first distal surface of the first monolithic lens and inoptical alignment therewith; and a second monolithic lens having secondproximal and distal surfaces, wherein the second proximal surface of thesecond monolithic lens has a plano profile, and wherein the seconddistal surface of the second monolithic lens has one of either a concaveor convex profile, and wherein the second monolithic lens is arranged inoptical alignment with said aperture; wherein the first monolithic lenselement is in direct contact with the aperture and the second monolithiclens.
 15. The array camera of claim 1, wherein the optics of each camerain the plurality of cameras comprise a three-element monolithic lensoptical arrangement comprising: a first lens element having a firstconvex proximal surface and a first plano distal surface; a second lenselement having a second concave proximal surface and a second convexdistal surface; a third menisci lens element having a third concaveproximal surface and a third convex distal surface; at least oneaperture disposed on the first plano distal surface; and wherein thefirst, second and third lens elements are arranged sequentially inoptical alignment with the aperture stop and an imager.
 16. The arraycamera of claim 1, wherein the camera array is a monolithic integratedmodule comprising a single semiconductor substrate on which all of thesensor elements are formed, and optics including a plurality of lenselements, where each lens element forms part of the separate optics forone of the cameras.
 17. The array camera of claim 1, wherein each of thecameras includes one of a plurality of different types of filer.
 18. Thearray camera of claim 1, wherein cameras that include different types offiler operate with different operating parameters.
 19. The array cameraof claim 17, wherein cameras having the same type of filter areuniformly distributed about the geometric center of the camera array.20. An array camera, comprising: a plurality of cameras, where eachcamera includes separate optics, and a plurality of light sensingelements; a processor; wherein the optics of each of the plurality ofcameras are configured so that each camera has a field of view that isshifted with respect to the field-of-views of the other cameras and sothat each shift includes a sub-pixel shifted view of the scene; whereinthe light sensing elements of a given camera in the plurality of camerashave a pixel pitch defining a camera Nyquist frequency, and where theoptics of the given camera have a modular transfer function (MTF) suchthat the optics optically resolve contrast at spatial frequencies higherthan the camera Nyquist frequency (Ny); and wherein software directs theprocessor to: obtain a set of low resolution images from the pluralityof cameras, where each of the low resolution images includes differentaliasing patterns due to the different sub-pixel shifted views of thescene provided by the optics of the plurality of cameras; determinedisparity between pixels in the set of low resolution images to generatea depth map from a reference viewpoint, where the depth map indicatesdistances to surfaces of scene objects from the reference viewpoint; andsynthesize a high resolution image using the set of images and the depthmap by: determining scene dependent geometric corrections to apply tothe pixels from each of the images within the set of low resolutionimages to eliminate disparity; fusing the set of low resolution imagesusing the scene dependent geometric corrections; and performing superresolution processing to reconstruct the high resolution image using thefused image, the scene dependent geometric corrections, and the set oflow resolution images; wherein the spatial frequency at which the highresolution image displays contrast is greater than the camera Nyquistfrequencies (Ny) of the plurality of cameras and less than the spatialfrequencies at which the optics of each camera optically resolvecontrast.